So apparently, the Internet Engineering Task Force is going to introduce a new HTTP status code. Just like there's the 404 for "File not found", we're soon going to have "397 Tolerating", similar to a redirect.
The way it would work is, if you send a request that violates some standard, but the server can identify the probable intent of your request, it will reply with a "397 Tolerating" to say, "oh, you messed up, and here's how, but I'm going to reply to what I think you meant".
This is much better than the options we had before, which were either a) unnecessarily reject the request, or b) silently reply to the intended meaning but with no notice that was happening. This lets you tell the client you're tolerating their garbage!
My contact at IETF send me an early draft of the RFC, which you can access at the link below.
RFC 8969: HTTP Status Code 397: Tolerating
Pastebin Link
A blog by Silas Barta
Showing posts with label engineering. Show all posts
Showing posts with label engineering. Show all posts
Wednesday, April 1, 2020
Tuesday, December 8, 2009
World's newest space agency: Reuters
I normally don't pay much attention to photo credits, but I had to do a double-take on this one. An article in the Telegraph has a satellite picture of the sun. Of course, to get that kind of picture, you have to get pretty close, exist in a high temperature environment, and have photography equipment capable of significantly attenuating the EM radiation thrown off from the sun
And who do they credit for the photo? NASA, right? No, we get:
Um, yeah dude. I think Reuters got that photo from someone else. With the budget cuts the media have had to make in the past few years, they can only afford near-earth satellites. Deeper-space probes are just out of the question.
ANYWAY, since I haven't posted on Climategate, or for that matter, anything in a while, here are my thoughts: It's absolutely disgraceful, the way the scientists in question have acted. Disclosure of your data does not mean that skeptics get to go on a multi-year scavenger hunt to find your raw data and then play guessing games about which sources you threw out and why.
The very fact that you have to make a post like this one in order to summon forth all the data is proof that you weren't being transparent enough.
There's also clear evidence that the scientists didn't seem to understand that you can't contort one data source to look like another and then call it two independent sources of data. Eric S. Raymond has done a tremendous job at exposing the tricks in the code, which explains exactly why the insular climate science doesn't want critics poring over their work
Oh, and just a hint: when you only allow people you approve of to review your work, that's not science.
PS: Recall that my outrage at many libertarians has been to their reactions *conditional* on AGW being real, and that outrage remains.
And who do they credit for the photo? NASA, right? No, we get:
Professor Henrik Svensmark argued that the recent warming period was caused by solar activity. Photo: REUTERS
Um, yeah dude. I think Reuters got that photo from someone else. With the budget cuts the media have had to make in the past few years, they can only afford near-earth satellites. Deeper-space probes are just out of the question.
ANYWAY, since I haven't posted on Climategate, or for that matter, anything in a while, here are my thoughts: It's absolutely disgraceful, the way the scientists in question have acted. Disclosure of your data does not mean that skeptics get to go on a multi-year scavenger hunt to find your raw data and then play guessing games about which sources you threw out and why.
The very fact that you have to make a post like this one in order to summon forth all the data is proof that you weren't being transparent enough.
There's also clear evidence that the scientists didn't seem to understand that you can't contort one data source to look like another and then call it two independent sources of data. Eric S. Raymond has done a tremendous job at exposing the tricks in the code, which explains exactly why the insular climate science doesn't want critics poring over their work
Oh, and just a hint: when you only allow people you approve of to review your work, that's not science.
PS: Recall that my outrage at many libertarians has been to their reactions *conditional* on AGW being real, and that outrage remains.
Wednesday, November 19, 2008
My plan to destroy the universe won't work
And I'll bet you're relieved!
Maybe a little background is in order.
A question of interest to philosophers and theoretical physicists is whether or not the universe is just a simulation running on some computer, one level up. (See e.g. Nick Bostrom's Simulation Argument.) Of course, many ridicule this idea as being non-falsifiable and thus non-scientific.
Not so fast! I said. Of course it's falsifiable. Here's how: if the universe is a simulation, then its programmers probably try to economize on computational resources (computing cycles, memory, disc space, time, etc). And to do that, they will make the program reveal to "us" (the conscious entities) the minimum required to make everything appear "believable". That in turn, means that as long as we "wouldn't know the difference" if some physical process developed in a way contradicting the rest of our observations, the simulator won't bother to churn through the calculations needed to make the process match up with known universal laws. In other words: "If we're not looking, why bother making sure something's there?"
And that tells us how to test the Simulation Hypothesis: have everyone set up as much measurement equipment as they can, and therefore observe as much as they can. This will force the simulator do many more calculations than it would otherwise have to, since now it has to keep consistent with that many more observations. The programmers then have to devote an ever-increasing amount of resources to keep it running, which will eventually force them to "cut corners" in implementing the laws of physics, revealing violation of Standard Model physics, or ... um, make them pull the plug on our existence.
Hence, my "plan to destroy the universe".
Now, the good news: the plan wouldn't work, based on what we already know about how the universe would react to such a "hypermeasurement" scenario! And the reason is shocking: because we can't actually increase our total knowledge.
"What in the hay-ll? I did me some book-larnin' not but three yurs ago!"
Sorry, that was Cletus, our resident country bumpkin.
Well, I'll need to some more background now to justify that claim. First, I want to point you to a post on OvercomingBias.com that introduced me to a lot about what I'll discuss here: Engines of Cognition.
Now, consider the 2nd law of thermodynamics. There are many ways to express it, but a simpler way is: "The amount of disorder ('entropy') in the universe must always increase." Sure, you can increase the order any one specific place -- say, when you form crystals -- but it will always be counterbalanced by an increase in disorder somewhere else. The most common application of this law is in heat engines (such as the one in your car): when you burn fuel to turn your engine and thus your tires, you are extracting a kind of order: the useful mechanical "work" (as it is called in physics) of a spinning engine. However, to do so, you burn fuel and transfer heat to the environment, which, when tabulated, generates entropy/disorder exceeding that which you destroyed in extracting mechanical work from the system to drive.
Now, here's the kicker: there are deep parallels between the concept of entropy in thermodynamics, and the concept called "entropy" in information theory. In the latter, it refers (roughly) to the uncertainty one has about the content of a message before reading it. Any knowledge that some kinds of messages are more likely than others therefore reduces that "entropy". Similarly, entropy is at a maximum when all messages are equally likely.
And the truly mind-blowing part is that the connection between the two kinds of entropy is so deep that entropy in the information-theoretic sense affects entropy in the thermodynamic sense. (This is going somewhere, just be patient.) In short, if you are able to reduce your uncertainty (information-theoretic entropy) about the "message" contained in the molecules of a system, that knowledge can actually be exploited to reduce the thermodynamic entropy of the system and thereby extract useful work! (For reference, and early exploration of this idea is called the Maxwell's Demon thought experiment, and a hypothetical engine that extracts work this way is the Szilard engine.)
But this hypothetical capability of decreasing the entropy of a system does not actually contradict the 2nd Law, which, you'll remember, says that total entropy must increase. Rather, for reasons I won't go into, this acquisition of knowledge itself is limited by the 2nd Law. Just as the extraction of "organized" mechanical work from fuel requires the generation somewhere else, of at least as much counterbalancing disorganization, so too does the collection of information that could permit extraction of the same work without the fuel require a counterbalancing loss of information somewhere else, i.e. increased uncertainty.
This principle reveals a fundamental limit that your brain (in a deep sense, a "cognitive engine") faces: in order to learn something true about your environment (whether via the senses or inferences), you must sacrifice knowledge somewhere else. Fortunately, nothing requires you to care much about that lost knowledge, which takes the form of "lost certainty about aggregate statistical properties of thermodynamic variables".
Now, back to the main point: from the perspective of hypothetical beings running the universe's simulator, my idea to gather more measurements has no impact. Any time we make a measurement, we are gathering knowledge, which must therefore correspond to lost knowledge somewhere else. So, far from threatening the computer's ability to simulate our universe, all our measurements will (amazingly) decrease the computational resources the simulator requires.
Which neatly returns the Simulation Hypothesis to non-falsifiability, and assures us that even if people acted on my idea, we're still safe and sound. Alternatively, it reveals the universe's programmers to be really, really clever :-)
Maybe a little background is in order.
A question of interest to philosophers and theoretical physicists is whether or not the universe is just a simulation running on some computer, one level up. (See e.g. Nick Bostrom's Simulation Argument.) Of course, many ridicule this idea as being non-falsifiable and thus non-scientific.
Not so fast! I said. Of course it's falsifiable. Here's how: if the universe is a simulation, then its programmers probably try to economize on computational resources (computing cycles, memory, disc space, time, etc). And to do that, they will make the program reveal to "us" (the conscious entities) the minimum required to make everything appear "believable". That in turn, means that as long as we "wouldn't know the difference" if some physical process developed in a way contradicting the rest of our observations, the simulator won't bother to churn through the calculations needed to make the process match up with known universal laws. In other words: "If we're not looking, why bother making sure something's there?"
And that tells us how to test the Simulation Hypothesis: have everyone set up as much measurement equipment as they can, and therefore observe as much as they can. This will force the simulator do many more calculations than it would otherwise have to, since now it has to keep consistent with that many more observations. The programmers then have to devote an ever-increasing amount of resources to keep it running, which will eventually force them to "cut corners" in implementing the laws of physics, revealing violation of Standard Model physics, or ... um, make them pull the plug on our existence.
Hence, my "plan to destroy the universe".
Now, the good news: the plan wouldn't work, based on what we already know about how the universe would react to such a "hypermeasurement" scenario! And the reason is shocking: because we can't actually increase our total knowledge.
"What in the hay-ll? I did me some book-larnin' not but three yurs ago!"
Sorry, that was Cletus, our resident country bumpkin.
Well, I'll need to some more background now to justify that claim. First, I want to point you to a post on OvercomingBias.com that introduced me to a lot about what I'll discuss here: Engines of Cognition.
Now, consider the 2nd law of thermodynamics. There are many ways to express it, but a simpler way is: "The amount of disorder ('entropy') in the universe must always increase." Sure, you can increase the order any one specific place -- say, when you form crystals -- but it will always be counterbalanced by an increase in disorder somewhere else. The most common application of this law is in heat engines (such as the one in your car): when you burn fuel to turn your engine and thus your tires, you are extracting a kind of order: the useful mechanical "work" (as it is called in physics) of a spinning engine. However, to do so, you burn fuel and transfer heat to the environment, which, when tabulated, generates entropy/disorder exceeding that which you destroyed in extracting mechanical work from the system to drive.
Now, here's the kicker: there are deep parallels between the concept of entropy in thermodynamics, and the concept called "entropy" in information theory. In the latter, it refers (roughly) to the uncertainty one has about the content of a message before reading it. Any knowledge that some kinds of messages are more likely than others therefore reduces that "entropy". Similarly, entropy is at a maximum when all messages are equally likely.
And the truly mind-blowing part is that the connection between the two kinds of entropy is so deep that entropy in the information-theoretic sense affects entropy in the thermodynamic sense. (This is going somewhere, just be patient.) In short, if you are able to reduce your uncertainty (information-theoretic entropy) about the "message" contained in the molecules of a system, that knowledge can actually be exploited to reduce the thermodynamic entropy of the system and thereby extract useful work! (For reference, and early exploration of this idea is called the Maxwell's Demon thought experiment, and a hypothetical engine that extracts work this way is the Szilard engine.)
But this hypothetical capability of decreasing the entropy of a system does not actually contradict the 2nd Law, which, you'll remember, says that total entropy must increase. Rather, for reasons I won't go into, this acquisition of knowledge itself is limited by the 2nd Law. Just as the extraction of "organized" mechanical work from fuel requires the generation somewhere else, of at least as much counterbalancing disorganization, so too does the collection of information that could permit extraction of the same work without the fuel require a counterbalancing loss of information somewhere else, i.e. increased uncertainty.
This principle reveals a fundamental limit that your brain (in a deep sense, a "cognitive engine") faces: in order to learn something true about your environment (whether via the senses or inferences), you must sacrifice knowledge somewhere else. Fortunately, nothing requires you to care much about that lost knowledge, which takes the form of "lost certainty about aggregate statistical properties of thermodynamic variables".
Now, back to the main point: from the perspective of hypothetical beings running the universe's simulator, my idea to gather more measurements has no impact. Any time we make a measurement, we are gathering knowledge, which must therefore correspond to lost knowledge somewhere else. So, far from threatening the computer's ability to simulate our universe, all our measurements will (amazingly) decrease the computational resources the simulator requires.
Which neatly returns the Simulation Hypothesis to non-falsifiability, and assures us that even if people acted on my idea, we're still safe and sound. Alternatively, it reveals the universe's programmers to be really, really clever :-)
Wednesday, September 10, 2008
So I was right again. Now, let's fix inflation measures.
There's a story on CNN Money today about shrinking and degrading products in response to inflation. Unfortunately, it doesn't give more than passing mention to the real stickler in inflation, the "degrading" part, which is harder for measurers to notice.
You don't say! I've been noticing this for a while, and haven't been convinced the BEA and BLS capture the impact. When you pay the same for a debased product, that is price inflation, and precisely what you need to measure. But like the fool who won't search for his keys outside of the light, the BEA and BLS don't do the lab testing necessary to incorporate critical quality-related aspects of products.
In my personal experience, I have noticed cereal boxes and paper cups as being flimsier and thus harder to hold -- about as big an inconvenience as you can tag onto such a simple, trivial product. Soda bottles also had confoundingly irritating changes: in addition to the 25% vending machine price increase, they shrunk the cap height beyond all reason so that it's nearly impossible to get a good enough grip to twist open with your hands. The fact that Coca-Cola even made this decision is a testimony to either a) the low quality of their engineering teams, or b) how desperately they needed to debase the product. Neither is encouraging. (To their credit, the caps have returned to "good enough", meaning they've hidden the price increase somewhere else.)
I should feel fortunate to live in a country where "difficulty in opening products" ranks highly enough to complain about. But that's also worrying: in a country with such enormous, overflowing wealth (which the US has, right?) shouldn't producers have kept such noticeable inconveniences out as a matter of course? Something's not right about that picture...
So, if you really want to measure inflation, you're going to have to track these very tricky quality changes. But there's an alternative: focus on measures were this quality debasement just isn't possible. As I'm sure I've argued here and on several boards by now, the ideal candidate is an insulin index which does the work of policing quality improvements for you. If you debase insulin, someone dies. The other benefits are:
-Steady, predictable demand
-Global market with many buyers
-Many inputs, so it's immune to any one specific input's volatility
-No transient intellectual property effects
Which probably accounts for why such information is so durn hard to find!
Second, in addition to capturing quality degradations, they need to fundamentally rework how luxury-type items are accounted for. Those typically "scale" with what other people have. Faster computers mean enabling nicer software, but they can also mean having to pay for hardware I don't need, as the older stuff isn't available, and my current one can't run the latest software that assumes I have a faster machine. And the value I can squeeze out of it doesn't increase one-to-one with the MegaHertz rating!
I absolutely accept that modern technologies have vastly expanded the entertainment and learning options available to me, but an inflation measure must at the same time account for when food and energy prices put the squeeze on me.
I'd be interested in transforming these ideas into an academic paper, except there are a few things ahead on that list...
Consumers are discovering more air in their bag of chips, fewer sheets of paper towels on the roll, thinner garbage bags and even smaller squares of toilet paper. (emphasis mine)
You don't say! I've been noticing this for a while, and haven't been convinced the BEA and BLS capture the impact. When you pay the same for a debased product, that is price inflation, and precisely what you need to measure. But like the fool who won't search for his keys outside of the light, the BEA and BLS don't do the lab testing necessary to incorporate critical quality-related aspects of products.
In my personal experience, I have noticed cereal boxes and paper cups as being flimsier and thus harder to hold -- about as big an inconvenience as you can tag onto such a simple, trivial product. Soda bottles also had confoundingly irritating changes: in addition to the 25% vending machine price increase, they shrunk the cap height beyond all reason so that it's nearly impossible to get a good enough grip to twist open with your hands. The fact that Coca-Cola even made this decision is a testimony to either a) the low quality of their engineering teams, or b) how desperately they needed to debase the product. Neither is encouraging. (To their credit, the caps have returned to "good enough", meaning they've hidden the price increase somewhere else.)
I should feel fortunate to live in a country where "difficulty in opening products" ranks highly enough to complain about. But that's also worrying: in a country with such enormous, overflowing wealth (which the US has, right?) shouldn't producers have kept such noticeable inconveniences out as a matter of course? Something's not right about that picture...
So, if you really want to measure inflation, you're going to have to track these very tricky quality changes. But there's an alternative: focus on measures were this quality debasement just isn't possible. As I'm sure I've argued here and on several boards by now, the ideal candidate is an insulin index which does the work of policing quality improvements for you. If you debase insulin, someone dies. The other benefits are:
-Steady, predictable demand
-Global market with many buyers
-Many inputs, so it's immune to any one specific input's volatility
-No transient intellectual property effects
Which probably accounts for why such information is so durn hard to find!
Second, in addition to capturing quality degradations, they need to fundamentally rework how luxury-type items are accounted for. Those typically "scale" with what other people have. Faster computers mean enabling nicer software, but they can also mean having to pay for hardware I don't need, as the older stuff isn't available, and my current one can't run the latest software that assumes I have a faster machine. And the value I can squeeze out of it doesn't increase one-to-one with the MegaHertz rating!
I absolutely accept that modern technologies have vastly expanded the entertainment and learning options available to me, but an inflation measure must at the same time account for when food and energy prices put the squeeze on me.
I'd be interested in transforming these ideas into an academic paper, except there are a few things ahead on that list...
Subscribe to:
Posts (Atom)