tag:blogger.com,1999:blog-11451982477623059572024-03-19T03:46:40.144-05:00Setting Things StraightA blog by Silas BartaSilas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.comBlogger165125tag:blogger.com,1999:blog-1145198247762305957.post-77648997810354306832024-02-22T10:52:00.008-06:002024-02-23T11:11:47.027-06:00Setting short-selling straight; or, "But who let you short that?"You might not be aware, but I've been <a href="https://en.wikipedia.org/w/index.php?title=Short_(finance)&useskin=vector#Physical_shorting_with_borrowed_securities">short-selling</a> some cryptocurrencies. (I would have said "been making money short-selling cryptocurrencies", but ...)<br><br>
Often people ask some from of, "oh wow, what broker lets you do that?" It's actually an interesting misunderstanding, in that it misses a key insight:<br><br>
<b>You can short-sell any time you have a debt denominated in that asset.</b><br><br>
At that point, you are <i>short</i> the asset. You benefit from anything that makes that asset easier to obtain, and thus extinguish your debt.<br><br>
In the decentralized finance world, there are platforms (in my case, <a href="https://compound.finance/">Compound.finance</a>) that let you deposit some crypto asset A, and borrow some other crypto asset B. Once you do so, and sell B, you are now short-selling B!<br><br>
The concept applies more generally too: for example, if you owe your friend the favor of helping them move, then you are "short moving services" (because anything that makes moving services cheap, also makes your debt easier to service, at least because you have the option to satisfying by paying a service rather than doing it yourself).<br><br>
Also, if you borrow US dollars, and spend them, you are "shorting the dollar", although it's usually not talked about in these terms. (You hold a debt denominated in dollars, after having "sold" them for something else, which we generally refer to as "buying".) Although, there was an interesting case where people would borrow USDC (a crypto dollar substitute) and then find out they could be bought back for less than a dollar during the <a href="https://en.wikipedia.org/w/index.php?title=Collapse_of_Silicon_Valley_Bank&useskin=vector">Silicon Valley Bank collapse</a>. Thus making them "accidental short-sellers"!Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com0tag:blogger.com,1999:blog-1145198247762305957.post-34589895474017188302024-01-28T10:58:00.000-06:002024-01-28T10:58:44.705-06:00A serious paper on bits as Joules per Kelvin<p>My ramblings about thermodynamics aren't so off-base, it turns out!</p>
<p>Remember <a href="http://blog.tyrannyofthemouse.com/2009/03/another-interesting-thermodynamics.html">this one?</a> From 2009? Where I explained how Joules per Kelvin (energy per unit temperature) is a valid measure of information (or entropy, effectively "missing information"), which is normally measured in bits?</p>
<p>Well, now there's a <a href="https://arxiv.org/pdf/2401.12119.pdf">paper</a> that formalizes that idea and related ones. As the title ("Temperature as Joules per Bit") indicates, it looks at a rearranged version of the same insight (mine was "bits as Joules per temperature"). But, it also goes a lot deeper and derives thermodynamics <em>starting</em> from entropy to understand temperature, rather than the other way around, as is conventionally done.</p>
<p>Related thought: I remember back in that 2009 thermodyanmics/info theory frenzy, one of my goals was rederive the <a href="https://en.wikipedia.org/wiki/Carnot_heat_engine?useskin=vector#Carnot's_theorem">Carnot limit</a> based on information-theoretic considerations -- that is, show it as a simple implication of the <em>amount</em> of knowledge you have about a system in a case where <em>only</em> know the temperature difference. (Naturally, I assumed someone had already done this and tried to find it but it was very hard to google for.)</p>
<p>Background: The Carnot limit tells you the maximum amount of mechanical work ("useful energy") you can extract from heat -- like, through an engine -- and, as it turns out, it's a function of the ratio of absoute temperatures you're working between. You don't face this limit when extracting work from a flywheel (spinning disc with grooves). Inspired by an counterintuitive insight in an Eliezer Yudkowsky <a href="https://www.lesswrong.com/posts/QkX2bAkwG2EpGvNug/the-second-law-of-thermodynamics-and-engines-of-cognition?utm_campaign=post_share&utm_source=link">LessWrong post</a>, and my <a href="https://www.lesswrong.com/posts/QkX2bAkwG2EpGvNug/the-second-law-of-thermodynamics-and-engines-of-cognition?commentId=5PaNLorAAEt9Ez6oh">thoughts about it</a>, I figured you could draw a more direct line from "knowledge of a temperature difference" to "how much energy is extractable".</p>
<p>Now I'll give it a go with ChatGPT, and post my findings!</p>Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com0tag:blogger.com,1999:blog-1145198247762305957.post-24856779187461390052023-05-21T10:30:00.000-05:002023-05-21T10:30:47.695-05:00ChatGPT4 refines my argument about fish being bad swimming instructors<p>I finally splurged for a paid ChatGPT subscription! With access to GPT-4, I asked it to refine my <a href="http://blog.tyrannyofthemouse.com/2022/08/extended-shower-thought-fish-would-be.html">earlier argument</a> about fish being bad swimming instructors. Here's what it gave me:</p><p>*** <br /></p><p>Certainly, let's rework the argument using more scientifically grounded explanations and references:<br /><br />While it's true that fish, with their adept aquatic abilities, may seem like logical swimming instructors for humans at first glance, upon closer examination, it becomes clear that this is far from practical or effective. Even if we put aside the substantial communication and physical barriers, there are fundamental biological and behavioral differences between fish and humans that pose significant challenges.<br /><br />Humans are terrestrial, bipedal mammals, and our motor skills and locomotive mechanisms are naturally adapted to this lifestyle. Fish, in contrast, are aquatic creatures designed to maneuver efficiently in water (Webb, P.W., 1982, "Locomotor patterns in the evolution of actinopterygian fishes"). Their ability to swim is an innate behavior, hardwired into their neural circuitry, not a learned skill (Gould, J.L., 1982, "Ethology: The Mechanisms and Evolution of Behavior"). Given this, fish cannot consciously comprehend or articulate the mechanics of their swimming abilities as humans would require in a teaching-learning scenario.<br /><br />In the field of pedagogy, it is well accepted that the process of teaching involves more than simply demonstrating a skill; it requires an understanding of learning styles, the ability to give and receive feedback, and a degree of self-awareness in the teacher (Boud, D., et al., 1985, "Reflection: Turning Experience into Learning"). Fish, as far as current scientific consensus suggests, lack these cognitive capacities (Butler, A.B., et al., 2005, "Evolution of the Neural Basis of Consciousness: A Bird-Mammal Comparison").<br /><br />Furthermore, many human skills, including swimming, are learned through a combination of instruction, practice, and feedback (Kirschner, P.A., et al., 2006, "Why Minimal Guidance During Instruction Does Not Work"). This model of learning doesn't align with the unconscious, reflex-driven behaviors seen in fish.<br /><br />Lastly, it's important to acknowledge the vast physiological differences between humans and fish. Human limbs and fish fins have evolved for different purposes and environments, with humans developing more for grasping and manipulation and fish for efficient aquatic locomotion (Thorsen, D.H., et al., 2004, "Biomechanics of Locomotion in Asian Carp"). These differences are fundamental and cannot be bridged by simple observation or imitation.<br /><br />In conclusion, despite their swimming prowess, fish cannot be effective swimming instructors for humans due to biological, behavioral, and cognitive differences, as well as practical communication and anatomical obstacles. This argument does not discount the value of observing aquatic creatures to improve our understanding of efficient movement in water, but it's clear that such observation is a far cry from active instruction. <br /></p>Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com1tag:blogger.com,1999:blog-1145198247762305957.post-46225999593747937532022-08-25T14:11:00.000-05:002022-08-25T14:11:15.617-05:00Extended shower thought: Fish would be bad swimming instructors<p>Obligatory <b>epistemic status</b>: Speculative and hard to test but falls out an as implication under current worldmodel.</p><p>Just a shower thought I've had on my mind for a while:</p><h3 style="text-align: left;">Fish would be bad swimming instructors (for humans).</h3><p>This remains true even after adjusting for/overcoming the differences in body types and ability to speak your language.</p><p>What do I mean? Well, to be sure, fish are <i>good</i> at swimming. And normally, that's what's you want in a swimming instructor...</p><p>...but not here. While fish are good at swimming, they have <i>always</i> been good at swimming. They were deft swimmers before they reached any point of being able to reflect on their selves or what they are doing. They have no appreciation for what it's like <i>not</i> to be a good swimmer.</p><h3 style="text-align: left;">What does "learning swimming" involve (for humans)? <br /></h3><p>Fish, being fish, never had to make the transition from being a natural biped to one that has repurposed its body for motion in water, which is what humans have to go through to "learn swimming". They never had to start from a mentality that finds walking somewhat natural -- and swimming somewhat unnatural -- and then adjusts its appendage motion in a way more suited to the latter.</p><p>We can imagine any kind of instruction session between fish and humans (as above, appropriately adjusted for language) to be fraught with peril. The fish might try to point out deficiencies that hold back the human. But it has never had to think about such deficiencies, because it has never needed to distinguish between the right way and the wrong way. It was always just "the way" that came instinctually.</p><p>The fish might nearly lash out, frustrated that the human attempts an obviously ineffective means of water locomotion. But it has no idea why those flawed attempts are wrong. They just <i>feel</i> wrong.</p><p>The fish will happily demonstrate the "right way", just by <i>doing</i> it. But identifying the difference between its "right way" and "what humans do" would be a learning experience in itself, something it has no natural advantage in, as it was never part of the fish's "learning process for swimming".</p><p>The converse is true -- humans would be bad at teaching fish to walk (with the appropriate apparatus) or otherwise move on land. (But note the parallel isn't perfect as humans generally do have to struggle and learn how to walk, but not in any way that involves formal instruction.)<br /></p><h3 style="text-align: left;">Implications for human-human interaction</h3><p style="text-align: left;">So far, this insight feels (and, well, is) just idle speculation. But there are implications for ordinary life.</p><p style="text-align: left;">There will be times when someone is a "natural" at some skill. They're good at it. And that is their only teaching qualification. And they try to teach a non-natural that skill. They are then bad at teaching. The missing piece is the learner's mind is very different from what the natural is capable of filling in. Such a teacher will constantly <i>show them</i> how to do it "right" but not be able to identify the difference between what they're doing and what the learner is -- except by drawing on some kind of unrelated, general intelligence.</p><p style="text-align: left;">(I won't give any specific examples of such a skill, because those become contentious issues in their own right and detract from the general point. But I've definitely been on both sides -- learning from someone who has no understanding why they're good at it, and grasping to communicate something I do without ever thinking about it.)</p><p style="text-align: left;">You can also run into this problem when you become skilled at something. You can sometimes assimilate the skill so well that you are effectively a natural, by forgetting the whole process by which you learned it. You may lose the perspective you had as a beginner and are no longer able to relate to them.</p><p style="text-align: left;">Part of why I enjoy teaching what I know is that I seem unusually resistant to this process, and have vivid memories of the hurdles I overcame as a newbie.<br /></p>Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com2tag:blogger.com,1999:blog-1145198247762305957.post-4448720817712476972021-08-15T12:48:00.003-05:002021-08-15T12:49:46.232-05:00Refuting arguments no one cares about: Exploiting the "help" in the pandemic vs the wildfiresI don't think anyone actually wanted to learn about this, but it's stayed on my mind for being "an argument that's wrong and I can prove".<br><br>
<b>Background:</b><br><br>
A while ago I made fun of a Facebook friend (call her D) for her "let them eat cake" cluelessness. This was during the 2018 (?) California wildfires when she made a big post saying how thankful she was that, in miserable air conditions, she could "still get her groceries" through delivery services. That prompted me (and, among others, another FB friend, "E") to drop our jaws and say, "Um, you don't care that you're just offloading all that suffering to poor workers that can't afford to just stay home, and will be breathing the lung-destroying air in your stead?"<br><br>
(And, to be sure, there's the argument that someone total utility is increasing by virtue of how said workers still have the option to expose themselves to risks for money, and the counterargument about "well why have OSHA that workers can't opt out of ..." which is its own topic but didn't really appeal to me or E at the time.)<br><br>
So far, so good.<br><br>
But later, during the pandemic, it came out that E, for similar reasons, didn't feel comfortable just getting delivery so she could stay home when we were being asked/mandated to.<br><br>
Those ... situations don't seem analogous at all to me, and I don't think someone should feel bad for ordering delivery during a pandemic like in an air quality emergency like wildfires. Here's why:<br><br>
A) In a wildfire, you are shifting all of the hazard of the smoke onto the people who bring your deliviers.<br><br>
B) But in a pandemic, moving to delivery reduces the hazard for <i>everyone</i>, including the delivery people.<br><br>
In Kantian terms: If "everyone did it", then everyone would still benefit in case B). But in A), all the avoidance by the rich "D" personas would be matched by losses to those who still have to deliver.<br><br>
To elaborate on B: the way a delivery service works, every worker involved has less Covid-spreading contact than than if everyone were shopping at a grocery store. The warehouses that set up the goods for delivery can, for their part, refactor and apply inexpensive countermeasures to reduce those worker's exposure. Furthermore, with everyone moving to delivery, you get economies of scale, allowing everyone to afford the delivery service.<br><br>
So, to me, it didn't didn't seem like you were doing anyone a favor out of solidarity to keep getting your grocerys through in-person shopping.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com1tag:blogger.com,1999:blog-1145198247762305957.post-39488943145042592232021-04-01T00:42:00.002-05:002021-04-02T13:46:22.972-05:00Leaked Google initiative: No more passwords!I have an inside source that's claiming Google will be rolling out a new replacement for passwords and other secrets for authenticating users. They shared the upcoming blog post/press release with me. They're moving to a more "holistic" authentication system? Let's see if this pans out. In any case, here's the not-yet-released announcement.<br /><br />
***
<h3>Are you who you claim to be?</h3><br/>
User logins protect websites from malicious actors, like spammers and trolls. So when you go online, only people with legitimate credentials can access the useful features of the site -- and others can't impersonate you. For years, you've used logins -- such as a username and password -- to prove to the site that you are who you claim to be, like this:</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo4UQ1NpTzynKcP2Kehrw9_u2Cg1wYl-L0-rrg2sZ1crPDcsnuMFbc6K64vJz1LUYOAfagcl8KWp1W512FV-uANwSTp95geeWRq3o3hvZ2lkNbggJefJP7v1oDBRs9X8WfIbPwIYqa9OHm/s494/normal_login2.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="279" data-original-width="494" height="181" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjo4UQ1NpTzynKcP2Kehrw9_u2Cg1wYl-L0-rrg2sZ1crPDcsnuMFbc6K64vJz1LUYOAfagcl8KWp1W512FV-uANwSTp95geeWRq3o3hvZ2lkNbggJefJP7v1oDBRs9X8WfIbPwIYqa9OHm/w320-h181/normal_login2.png" width="320" /></a></div><br /><br />
Some go even further and add a <a href="https://en.wikipedia.org/wiki/Multi-factor_authentication">second factor to authenticate with</a>, like an SMS code or one-time-password generator like you might have in the Google Authenticator app.<br /><br />
But, we figured it would be easier to just directly <i>ask</i> our users who they are -- so, we did! Following on our earlier success with <a href="https://security.googleblog.com/2014/12/are-you-robot-introducing-no-captcha.html">No CAPTCHA reCAPTCHA</a>, we’ve begun rolling out a new API that radically simplifies the login experience. We’re calling it "Credential-Free Authentication" and this is how it looks:<br /><br />
<div class="separator" style="clear: both;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLgc7tkR43ZFkwQKA741wRPgGMCCHXp5TZq_kPLhyIm_kkBFMB37z2PEMIqginXhc28UvtAcLEdtt899c08Fe6fzP-BLjzuhdEpS5e_tXY8lUCp2Ri2Dt7yAZvVf-Zm1VZdpjHgEr3Fa9-/s0/CFA_login3.png" style="display: block; padding: 1em 0; text-align: center; "><img alt="" border="0" data-original-height="379" data-original-width="560" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjLgc7tkR43ZFkwQKA741wRPgGMCCHXp5TZq_kPLhyIm_kkBFMB37z2PEMIqginXhc28UvtAcLEdtt899c08Fe6fzP-BLjzuhdEpS5e_tXY8lUCp2Ri2Dt7yAZvVf-Zm1VZdpjHgEr3Fa9-/s0/CFA_login3.png"/></a></div>
On websites using this new API, a significant number of users will be able to securely and easily verify their identities without (separately) having to provide credentials: no password, no rotating code. Instead, with just a single click, they’ll validate who they claim to be.<br /><br />
<h3>A brief history of user authentication</h3><br />
While the new login API may sound simple, there is a high degree of sophistication behind that modest interface. Authentication has long relied on attackers not having critical secrets, like a password or random number generator seed or other private information. You may have heard the traditional formulation, that authentication requires you to provide <a href="https://www.cs.cornell.edu/courses/cs513/2005fa/NNLauthPeople.html">something you have, something you are, or something you know</a>.<br /><br />
However, our research recently showed that it's about as likely for the genuine user to be missing the credentials as it is for a malicious actor. How many times have you forgotten your password or encountered a bug with your password manager? (Not <a href="https://support.google.com/accounts/answer/6208650?hl=en">GPM</a>, of course!) Thus, challenging users for credentials is no longer a dependable test.<br /><br />
Furthermore, attackers are often able to steal user credentials, forcing providers to rely on a secondary layer of <a href="https://support.google.com/paymentscenter/answer/9003744">fraud identification</a>, so as to lock accounts when users behave suspiciously. You've seen this if you've ever had a credit card declined for an unusually large or remote purchase.<br /><br />
<h3>Introducing Credential-Free Auhentication</h3><br/>
That got our security engineers thinking: if we already have to analyze a user's behavior in order to catch account compromises, why not just use <i>that</i> as the authentication? It would cut two carrots with one knife! After all, an attacker <i>might</i> be able to guess your password or your credit card information, but they will <i>never</i> be able to mimic the full depth and breadth of how you interact with websites, from your browing history, to your cookies set, to the way you move your mouse.<br /><br />
Following the "No CAPTCHA" model above, we developed an Advanced User Analysis backend for logins that actively considers a user’s entire engagement with the the Internet to determine who that user is. This enables us to rely less on "Do you have the secret?" and, in turn, offer a better experience for users. Now, users can just click a radio button, and in most cases, they’re logged in. In fact, you'll rarely have to log in at all, because sites will "recognize" you, just like you don't have to show your ID to go into an event venue a second time if the bouncer recognizes you.<br /><br />
<h3>But are you really that person?</h3><br/>
However, authentication challenges aren't going away just yet. In cases where our tracking cookies and other behavioral metrics can't confidently predict who someone is, we will prompt the user for additional information, increasing the number of security checkpoints to confirm who the user really is. For example, you might need to turn on your webcam or upload your operating system's recent logs to give a fuller picture.<br /><br />
<h3>Adopting the new API on your site</h3><br />
As more websites adopt the new API, more people will see Credential-Free Authentication. Early adopters, like Snapchat, WordPress, Twitch, and several others are already seeing great results with this new API. For example, in the last week, the number of support tickets for account resets on WordPress went down by 90%. Twitch reported similar figures -- and also was able to unmask several <a href="https://en.wikipedia.org/wiki/Sock_puppet_account">sockpuppets</a> who had been manipulating discussions and vote totals.<br/><br/>
To adopt the new CFA API for your website, visit our <a href="https://www.google.com/credentialfree">landing page</a> for more.<br/><br/>
Good users, we'll continue to keep the internet safe and easy to use. Bad users, it'll only get harder to hide yourselves and take over legitimate accounts -- sorry we're (still) not sorry.<br/><br/>
***<br/><br/>
Edit: Yes, this was an April Fools joke.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com3tag:blogger.com,1999:blog-1145198247762305957.post-72124473372448915072020-11-07T10:59:00.002-06:002020-11-07T11:32:49.520-06:00How they handle multi-episode stories in a TV series<b>Related to:</b> <a href="https://www.lesswrong.com/posts/HLqWn5LASfhhArZ7w/expecting-short-inferential-distances">Expecting Short Inferential Distances</a> (yes, linking LW is still a thing).
<br><br>
Below are some remarks I've made in scattered form in several forums, but I thought I'd collect them here.
<br><br>
<b>Problem:</b> A TV series will run for many episodes, and the writers will want to build up a storyline over many of them, which is necessary for a satisfying payoff. But viewers don't necessarily watch it all at once and keep the whole thing in their heads.
<br><br>
I call this the "narrative equivalent" of expecting short inferential distances (see the link above) -- just as people are used to an explanation not requiring a lot of steps, they don't expect a given scene or episode to need a lot of previous viewing to understand.
<br><br>
So, how do writers solve this problem? Here are the four general ways, with examples (feel free to offer more TV series for a category!):
<br><br>
<b>A) Don't bother #1: Each episode is self-contained.</b>
<br><br>
This is known as the <a href="https://fistfulofwits.com/2013/05/18/take-five-episodic-vs-serial-works/">"episodic" approach</a>, as opposed to serial. Each episode can be understood without knowing anything about the preceding episodes, so there's no need to worry about this problem at all.
<br><br>
Examples: Star Trek (at least the original or TNG), South Park, The Simpsons (earlier seasons)
<br><br>
Downside: It's hard to feel investment when you know nothing will matter, that things will just return to where they were at the end. It also limits how much build-up (and corresponding payoff) the story can have.
<br><br>
<b>B) Don't bother #2: It's the viewers' job to keep up.</b>
<br><br>
Each episode just assumes you have the entire previous history in your head, maybe with some small reminders of previous relevancies for assistance.
<br><br>
Example(s): Game of Thrones
<br><br>
Downside: It only works for really devoted fans who will binge the whole thing and try on their own to stay up to speed.
<br><br>
<b>C) Formal recaps</b>
<br><br>
You've seen them: the narrator says, "Previously, on [this TV series]...", and then you get enough short clips to establish the relevant context for current episode.
<br><br>
Examples: 24 (after season 1), Battlestar Galactica, Burn Notice
<br><br>
Downside: A lot of people don't like them and think they're cheesy. (I've never understood this mentality, but there it is.) It also may force you to reveal what things from previous episodes will be relevant, taking away their surprise when revealed. There is a slight break in immersion since you have to go "out of the world" for them.
<br><br>
<b>D) In-world recaps</b>
<br><br>
Same as the previous, except you're not taken "out of the world" to do it; instead, there is a scene, within the story, that doubles as exposition of the things a formal recap would cover.
<br><br>
Examples: Breaking Bad, Bojack Horseman
<br><br>
Downside: It heavily constrains how you write the story and forces in pointless scenes that shatter immersion because they're retelling things -- and more slowly! -- the characters should know.
<br><br>
Well, there you have it. That about summarizes the different ways writers handle (or avoid handling) long storylines!Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com6tag:blogger.com,1999:blog-1145198247762305957.post-18643460320766897522020-04-01T03:46:00.002-05:002020-04-01T12:13:03.784-05:00An HTTP status code to say "you messed up but I'll handle your request anyway"So apparently, the Internet Engineering Task Force is going to introduce a new HTTP status code. Just like there's the 404 for "File not found", we're soon going to have "397 Tolerating", similar to a redirect.<br />
<br />
The way it would work is, if you send a request that violates some standard, but the server can identify the probable intent of your request, it will reply with a "397 Tolerating" to say, "oh, you messed up, and here's how, but I'm going to reply to what I think you meant". <br />
<br />
This is <i>much</i> better than the options we had before, which were either a) unnecessarily reject the request, or b) silently reply to the intended meaning but with no notice that was happening. This lets you tell the client you're tolerating their garbage!<br />
<br />
My contact at IETF send me an early draft of the RFC, which you can access at the link below.<br />
<br />
<a href="https://drive.google.com/file/d/15XRtf4EVu5hHutL_Lt2zZLYUqH0hNakz/view?usp=sharing">RFC 8969: HTTP Status Code 397: Tolerating</a><br />
<br />
<a href="https://pastebin.com/TPj9RwuZ">Pastebin Link</a>Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com8tag:blogger.com,1999:blog-1145198247762305957.post-21769703877178631722020-03-18T10:48:00.000-05:002020-03-18T10:48:07.786-05:00My presentation on using SAT solvers for constraint and optimization problemsBecause of the virus we had to hold <a href="https://www.meetup.com/austinpython/events/266446399/">this meetup</a> virtually, and I was slated to present there for the Evening of Python Coding. Since we made a recording of it, I can now share. Enjoy my not-ready-for-prime-time voice! (Yes, I need to update my profile picture ... badly.<br />
<br />
<iframe width="560" height="315" src="https://www.youtube.com/embed/748bG0JMDrk" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com3tag:blogger.com,1999:blog-1145198247762305957.post-83096754176732516242019-05-31T18:45:00.003-05:002019-05-31T18:48:36.934-05:00Solving my first (Ghidra) reverse engineering challengeI was pointed to <a href="https://www.root-me.org/en/Challenges/Cracking/ELF-x86-CrackPass">this challenge</a> by <a href="https://www.freecodecamp.org/news/how-i-solved-a-simple-crackme-challenge-with-the-nsas-ghidra-d7e793c5acd2/">this article</a>, and had heard about the NSA's new <a href="https://www.nsa.gov/resources/everyone/ghidra/">Ghidra reverse engineering tool</a>. I was able to solve it without reading much of the article! Here's what I did.<br />
<br />
<b>Setup:</b> They give you a compiled ELF binary that runs a program that prompts you for a password. It will tell you if you guessed correctly.<br />
<br />
I was going to use this as a change to learn the Ghidra tool, with help from the article.<br />
<br />
<b>First problem:</b> It's compiled to run on Linux Debian x86. I was doing it on a MacBook. So, it wouldn't run.<br />
<br />
I noticed that Ghidra offers you the option to export the binary as C code (which makes sense, as part of Ghidra's reverse engineering is to convert a binary into assembly and slightly-more-readable C-like code). So, I figured I could just compile that to run on my Mac.<br />
<br />
It didn't work though, since Ghidra exports an incomplete version of the code that uses some invalid types, and doesn't define all of its labels properly, which required a lot of manual work that was increasingly appearing like it would take too long.<br />
<br />
So I figured I'd just run the binary in a Docker container. I found <a href="https://hub.docker.com/r/i386/debian/">the Debian i386 version</a> and pulled it down. (I had a long side diversion here getting a setup so I could edit files on my machine that would appear as I want in the Docker container, but the details are uninteresting besides <a href="https://stackoverflow.com/a/24167546">this clever hack</a> for getting the "copy file" command into your docker one-liner.)<br />
<br />
So, I was able to run the binary.<br />
<br />
<b>Second problem:</b> Somehow, it thought I was using a "debuguer" -- which I would be, at some point, but is strange, because I wasn't using one yet:<br />
<br />
<blockquote class="tr_bq"><span style="font-family: "courier new" , "courier" , monospace;"><br />
Don't use a debuguer !</span></blockquote><br />
<br />
I looked through the decompiled code to see where it might be doing this and found:<br />
<br />
<blockquote class="tr_bq"><span style="font-family: "courier new" , "courier" , monospace;">lVar1 = ptrace(PTRACE_TRACEME,0,1,0);</span><br />
<span style="font-family: "courier new" , "courier" , monospace;">if (lVar1 < 0) {</span><span style="font-family: "courier new" , "courier" , monospace;"></span><br />
<span style="font-family: "courier new" , "courier" , monospace;">puts("Don\'t use a debuguer !");<br />
/* WARNING: Subroutine does not return */<br />
abort();<br />
} </span></blockquote><br />
Hm, okay, well that "if" statement maps to this part of the assembly:<br />
<br />
<blockquote class="tr_bq"><span style="font-family: "courier new" , "courier" , monospace;"><br />
0804868c 79 11 JNS LAB_0804869f<br />
</span></blockquote><br />
That is, according to <a href="http://ref.x86asm.net/coder32.html">this handy reference</a>, "Jump if not sign." Well, whatever "sign" is, I want it to do the opposite -- change "JNS" to "JS", so it "jumps if sign" and skips that block -- the one that gives the mean message and exits the program.<br />
<br />
I haven't figured out a good way to format that line, but 0804868c is the (hexadecimal) location within the binary, 79 11 is the hex version of the binary command itself, JNS the assembly term (just a mnemonic device) for that command, and LAB_0804869f is the argument passed to the command -- in this case, a label for where to jump to in the program.<br />
<br />
According to that spec, the JNS command maps to 79, and if I want it to be JS, I need to replace it with 78. But...<br />
<br />
<b>Third problem:</b> I don't have an easy way to edit binaries.<br />
<br />
A convenient way to look at them is as a hex(adecimal) dump in a <a href="https://en.wikipedia.org/wiki/Hex_editor">"hex editor"</a>, but I hadn't done that for a few months. Hex editors are useful because they show you the raw hex, the offset where it appears in the file, and, off to the side, the ASCII/text equivalent of that hex. Fortunately, I found <a href="https://vi.stackexchange.com/a/2237">this comment</a>, showing how to repurpose my text editor, vim, to double as an easy hex editor. I can just open it up, edit the hex values, save, and it updates everything.<br />
<br />
You can then run a utility, "gobjdump", to see the code as assembly and verify that e.g. JNS changed to JS as expected.<br />
<br />
Fortunately, when I ran that edited binary, it did what I wanted: not accuse me of using a debugger, and proceed with the rest of the program.<br />
<br />
The rest of the hurdles are similar: there's some if-test that can send the execution into a block that you don't want it to. Fortunately, this binary is structured ... stupidly, from a security perspective. It has that same kind of if-block for validating whether you entered the right password. Right afterward, if the check succeeded, it prints the password. Not your input, no -- it prints the correct password, which you can later validate on an unmodified run.<br />
<br />
So, it's just a matter of tweaking the code so that the you always enter the block that outputs the password. (They were smart enough not to have the password itself appear as an obvious string in the binary, at least.)<br />
<br />
Once I saw the output, I could submit it at the challenge site and very it was correct.<br />
<br />
Now for something harder!Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com3tag:blogger.com,1999:blog-1145198247762305957.post-50429319067330484662018-02-13T19:49:00.000-06:002018-02-13T19:49:02.132-06:00NAND to Tetris: Nothing short of awesomeBeen a while since I posted, huh?<br />
<br />
Well, here's an interesting development in my life: I finally started the <a href="https://www.coursera.org/learn/build-a-computer/home/welcome">NAND to Tetris</a> course (<a href="https://www.coursera.org/">part 2</a>) on <a href="http://coursera.org">Coursera</a>, which is based on the <a href="">instructors' site</a>.<br />
<br />
I can say, whole-heartedly, that the course is awesome, and the first thing in a long time that has gotten me into a <a href="https://en.wikipedia.org/wiki/Flow_(psychology)">flow state</a>.<br />
<br />
The idea behind the course is this: they take you through (virtually) building a computer from a very basic level, showing all the abstraction layers that produce a computer's behavior.<br />
<br />
It starts with logic gates -- specifically, the <a href="https://en.wikipedia.org/wiki/NAND_gate">NAND gate</a> (hence the name), since it's universal. Your first project is to use a hardware simulator to build several other logical gates from NAND. (NAND is just an AND gate with the outputs inverted, so that true and true yield false, everything else yields true.) The second project is to to build the arithmetic logic unit (ALU) in a CPU out of the components, so you basically have a configurable circuit: based on six control bits, you perform one of several possible functions on two 16-bit inputs.<br />
<br />
The next project incorporates "flip flops", which allow you to repeatedly execute that circuit while also writing to and reading from some some persisted memory. Your inputs to the circuit then function as a kind of machine code.<br />
<br />
The project after that then has you implement functions in that machine code, writing in an assembly language the authors created for the course that has a precise mapping to the machine code inputs in the CPU from the previous lesson. I can honestly say it was a really fun project to implement multiplication directly in assembly language!<br />
<br />
Later on you write a compiler from a high level language into assembly (which then converts simply into machine code), in a way that's broken into two steps: a compiler from the high level language into a virtual machine that works on a stack with memory blocks, and a compiler from the virtual machine commands to assembly. I just recently finished that latter part (which comes first).<br />
<br />
Those layers then build up to making a game that runs on your CPU, then an OS, and some other stuff I haven't delved into.<br />
<br />
In the course of the projects, you use a hardware simulator, an assembler (for converting the assembly language into machine code), a CPU simulator, and a virtual machine simulator.<br />
<br />
In order to catch up with the class an have enough leeway for a weekend trip, I went through much of the course over the weekend, and enjoyed every minute of it! I especially like how the break the projects into manageable pieces. For example, in the VM-to-assembly part, it first has you implement simple push/pop/add operations on a stack and run a test to verify that you can compile those commands. Further test suites give you a manageable set of operations to add.<br />
<br />
I've written an actual compiler now! (albeit limited use...)<br />
<br />
(Consider how fun this is, and how naturally it comes to me, maybe I picked the wrong field.)<br />
<br />
One interesting challenge is that the CPU only has two registers call A and D, and only the A register is used when accessing memory, to know which word of memory to look at. It took me a while to figure out how you could say "look at the value in the memory location refered to in the memory location zero." Before I saw how you could do it, I implemented one project by having two separate code branches for the two possible values at the memory location!<br />
<br />
Would love to link my code for the projects, but they discourage that to leave the challenge for others to solve.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com10tag:blogger.com,1999:blog-1145198247762305957.post-17609685653746575452016-11-21T11:33:00.000-06:002016-11-21T11:33:37.284-06:00Another Slashdot memory: Ah, the brick-and-mortar analogies...<br />
So, I remembered another funny Slashdot exchange (again, no link).<br />
<br />
Story: Some online retailer got in trouble for filtering their "customer reviews" so that only the positive remarks (about listed products) stayed and everything else was deleted.<br />
<br />
A: "Wow, that's pretty scummy. They don't have the right to just clip out negative reviews."<br />
<br />
B: "Don't they? I mean, it's their site; they have the right to set whatever editorial standards they want."<br />
<br />
C: "Sure, but there's still an issue of consumer fraud and deception. For a brick-and-mortar analogy, imagine that Barnes and Noble started hosing 'book discussion nights' at their stores and promoted it as such. But you quickly notice that whenever someone says something negative about a book sold by B&N -- and only those books -- that person gets a tap on the shoulder from security, pulled aside, and asked to leave.<br />
<br />
"In that case, it would indeed be correct to say they can expel whoever they want, but it's still fundamentally fraudulent to represent that event as being for 'book discussion' rather than 'book promotion'."<br />
<br />
<hr><br />
In other news, I finally found one of the ones that I thought I couldn't! The Armadillo rocket failure mentioned in <a href="http://blog.tyrannyofthemouse.com/2016/01/funny-slashdot-exchanges-before-theyre.html">this post</a> was actually <a href="https://hardware.slashdot.org/comments.pl?sid=276831&cid=20322111">this conversation</a>. The actual (but truncated) exchange went like this (note the links to original comments):<br />
<br />
<a href="https://hardware.slashdot.org/comments.pl?sid=276831&cid=20322111">A</a>: "And to think, they want us all to ride in these things commercially...."<br />
<br />
<a href="https://hardware.slashdot.org/comments.pl?sid=276831&cid=20322611">B</a>: "John and his team have an excellent track record thus far, and have continued to make safety a main issue. I'm sure that this experience will teach them even more, helping to make the next flight even safer."<br />
<br />
<a href="https://hardware.slashdot.org/comments.pl?sid=276831&cid=20322893">C</a>: "You mean even safer than a huge orange fireball?<br />
<br />
"I don't know, that's a pretty high bar."Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com6tag:blogger.com,1999:blog-1145198247762305957.post-2590014222915030212016-02-27T16:04:00.001-06:002016-02-27T16:13:44.348-06:00Some of my geeky tech jokes -- with explanations!I know the line: explaining a joke is <a href="http://www.goodreads.com/quotes/440683-explaining-a-joke-is-like-dissecting-a-frog-you-understand">like dissecting a frog</a>; you understand it better, but it dies. Still, not everyone will get these, and I figure I might as well have a place where you at least get a chance. So here are some of my own creations, explained.<br />
<br />
<hr><br />
Girl, you make me feel like a fraudulent prover in a stochastic interactive zero-knowledge proof protocol ... because I really wish I had access to your random private bits!<br />
<br />
<b>Explanation:</b> In a stochastic zero-knowledge proof protocol, there is a prover and a verifier, where the former wants to convince the latter of something. But for proof to work, the verifier must give the prover unpredictable challenges. Think of it like a quiz in school -- it's not much of a quiz if you know the exact questions that will be on it.<br />
<br />
The information to predict the challenges is known as the verifier's <a href="http://www.scottaaronson.com/democritus/lec12.html">private random bits</a> Those with a legit proof don't need this, but a fraudulent prover does. Thus, a fraudulent prover in a stochastic interactive zero-knolwedge proof protocol wants access to the verifier's "random private bits".<br />
<br />
<hr><br />
A historian, a geologist, and a cryptographer are searching for buried treasure. The historian brings expertise on practices used by treasure hiders, the geologist brings expertise on ideal digging places, and the cryptographer brings expertise on hidden messages.<br />
<br />
Shortly after they start working together, the cryptographer announces, "I've found it!!"<br />
<br />
The others are delighted: 'Where is it?'<br />
<br />
The cryptographer says, "It's underground."<br />
<br />
'Okay, but where underground?'<br />
<br />
"It's somewhere underground!"<br />
<br />
'But where specifically?'<br />
<br />
"I don't know, but I know it's underground!"<br />
<br />
'Slow down there. If all you know is that it's underground, then in what sense did you "find" anything? We're scarcely better off than when we started!'<br />
<br />
"Give me a break! I just gave you an efficiently-computable distinguishing attack that separates the location of the treasure from the output of a random oracle. What more could you want?"<br />
<br />
<b>Explanation:</b> In cryptography, an encryption scheme is considered broken if an attacker can find some pattern to the encrypted message -- i.e. they can identify telltale signs that it wasn't generated by a perfect random number generator, a "random oracle". Such a flaw would be called a "distinguishing attack". So in the cryptography world, they don't care if the attack actually allows you to decrypt the message; they stop as soon as they find non-randomness to the encrypted data. Applied to a treasure hunt, this means they would give up as soon as they conclude that the treasure location is non-random, which the cryptographer here things s/he's done simply by concluding that it's "underground".<br />
<br />
<hr><br />
So, 16-year-old Johnnie walked into an Amazon Web Services-run bar...<br />
<br />
"Welcome," said the bartender. "What are you drinking?"<br />
<br />
Johnnie replied, 'What've you got?'<br />
<br />
"Well, we have a selection of wines and the beers you see right here on tap. But if you prefer, we also have club soda and some juices."<br />
<br />
Johnnie thought, Wait a second. Why is he telling me about the wines and beers? Does he even realize ... ?<br />
<br />
'Okay, I'll take the Guinness.'<br />
<br />
"Bottle or draft?"<br />
<br />
'Draft.'<br />
<br />
"Alright, and how will you be paying?"<br />
<br />
Johnnie only had large bills from his summer job and gave the bartender a C-note.<br />
<br />
"Sorry, but I gotta check to make sure this is real." The bartender took out a pen and marked it, then counted out the change. Johnnie reached for the beer.<br />
<br />
"Hold on a second! Make sure to use a coaster!" The bartender slipped one under the glass. "Okay, now enjoy!"<br />
<br />
Johnnie lifted up the glass to drink. Before he was able to sip, the bartender swatted it out of his hand.<br />
<br />
"WHAT ARE YOU THINKING!?! Don't you know 16-year-olds can't drink!"<br />
<br />
<b>Explanation:</b> On the AWS site, they will gladly let you click on the "Launch server" button and go through numerous screens and last-minute checks to configure it, and only at the very last stage does it say, "oops, turns out you don't have permission to do that" -- so it's like a bartender that takes you through a entire transaction, even verifying irrelevant things (like whether the money is real), while knowing the whole time he can't sell to you.<br />
<br />
<hr><br />
How is a Mongo replica set like an Iowa voter?<br />
<br />
In primary elections, they only vote for candidates they think are electable!<br />
<br />
<b>Explanation:</b> Databases can have "replica sets" where there are multiple servers that try to have the same data; secondary servers depend on an agreed-upon "primary" to be the "real" source of data. Often times, the primary server goes down, so they have to decide on a new primary, known as a "primary election". But there are some restrictions on who they will vote for -- if they e.g. have reason to believe that a server can't be seen by other members, and in those cases it will regard that server as unelectable. So you can get funny messages about "server42 won't vote for server45 in primary election because it doesn't think it's electable".Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com12tag:blogger.com,1999:blog-1145198247762305957.post-36619335012597273772016-02-20T19:31:00.000-06:002016-02-20T19:31:23.055-06:00More funny and insightful Slashdot posts I rememberIf you liked the <a href="http://blog.tyrannyofthemouse.com/2016/01/funny-slashdot-exchanges-before-theyre.html">last post on this</a> here are some more you might like. This time, I think they're more insightful than funny, but often times, <a href="http://www.brainyquote.com/quotes/quotes/i/isaacasimo109758.html">insight <i>is</i> funny!</a><br />
<br />
Also, I thought I'd point out the value of forums: in all of these exchanges, it seems to take three people to get the "aha" moment, not just one or two. <br />
<br />
<hr><br />
Topic: some new superstrong carbon nanotube material is discovered.<br />
<br />
A: So it seems like they're planning to use this stuff for armor. But what about weapons? It seems that anything strong enough to make good armor would also have value as a weapon.<br />
<br />
B: That doesn't follow at all! They're completely different use-cases. For example, leather is known to make good armor, but you never see a leather sword!<br />
<br />
C: Sure you do -- it's called a "whip" and we dig them up all the time!<br />
<br />
<div class="separator" style="clear: both; text-align: center;"><a href="http://i.imgur.com/gpLDGr7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://i.imgur.com/gpLDGr7.png" /></a></div><br />
<hr><br />
Topic: Some dating site is blocked from emailing University of Texas students (at their school email) because of too much spam.<br />
<br />
A: Well, their defense is that they're complying with the CAN-SPAM Act, in that they have an unsubscribe link, use their real name, etc., so UT can't just block them wholesale.<br />
<br />
B: What? That doesn't matter; that's just the *minimum* requirement for sending such emails. Obviously, any domain owner can impose whatever extra restrictions they want!<br />
<br />
C: Yeah, it's like saying, "I have a valid driver's license. I am the legal owner of this vehicle. I have paid the appropriate taxes and registered it. I hold the required liability insurance, and the vehicle is in good working order. I have complied with all applicable traffic laws and operated it in a safe manner. Therefore, I have the right to park on your lawn."<br />
<br />
<hr><br />
I actually use C's last example when explaining to people the difference between authentication (proving who you are) and authorization (what that identity is allowed to do).Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com1tag:blogger.com,1999:blog-1145198247762305957.post-83329021735147511522016-02-20T19:14:00.001-06:002016-02-20T19:14:38.031-06:00If the minimum wage prohibitions are so easily circumvented ...The recent <a href="https://medium.com/@taliajane/an-open-letter-to-my-ceo-fb73df021e7a#.aik8mpjep">Talia Jane story</a> just <a href="https://news.ycombinator.com/item?id=11142634">made me realize</a> we have a possible inconsistently in policy. To get you up to speed, Jane took a low-wage job in the San Francisco Bay Area, hoping to work her way up to her passion of being a social media manager for a major company. But because of rental prices, she paid 85% of per post-tax pay just for rent (!), complained about her employer paying so little, and then was fired.<br />
<br />
But as for the inconsistency:<br />
<br />
Illegal: paying someone below $X/hour.<br />
<br />
Legal: paying someone ($X + $Y)/hour (Y positive) to work in a place where their discretionary income would place them in extreme poverty (e.g. 85% of post-tax on rent).<br />
<br />
And yes, that's just an (arguably trivial) corollary of "minimum wage (and tax brackets for that matter) is not automatically cost-of-living-adjusted". But if the goal is to stop people from being taken advantage of with low job offers that hold them in poverty, that seems like a pretty big loophole.<br />
<br />
And it's not just that -- let's say someone moves farther out to be able to afford to live there. Then they're traveling an extra N hours just to make each shift which should rightly count against their effective hourly wage.<br />
<br />
So, food for thought: what are we really trying to optimize for here? What would the law have to look like to not just avoid these loopholes, but "carve reality at the joints" such that it's fundamentally impossible to scalably circumvent such a law?<br />
<br />
If you keep raising the minimum wage for a locality, and people keep commuting greater distances to get that income, what have you accomplished?Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com2tag:blogger.com,1999:blog-1145198247762305957.post-35225659935302615032016-01-07T21:31:00.000-06:002016-01-07T21:31:44.734-06:00Funny Slashdot exchanges, before they're lost to timeIn the time that I was a regular reader of <a href="http://slashdot.org/">Slashdot</a>, I saw a few exchanges that stayed in my mind. I later went back to find them, but was never able to. So that they're not lost to time, I figured I'd post all the ones I remember. What follows is from memory, and prettied up a bit. (Not trying to plagiarize, if you can find the original post for any of these, let me know.)<br />
<br />
Enjoy.<br />
<br />
<hr><br />
[Story: Armadillo Aerospace has a failed rocket launch.]<br />
<br />
A: Well, I think we can close the books on Carmack's little project.<br />
B: Come on, now. Private space travel is still in its infancy. There are growing pains. Not everything works the first time. But what's important, is that we're <i>learning</i> from these events. Armadillo is learning. They'll adapt. And the next voyage will be better and safer!<br />
C: You mean, even <i>safer</i> than a big orange fireball?<br />
<br />
<hr><br />
A: [long rant] So that's the problem with this ban on incandescent light bulbs.<br />
B: Whoa whoa whoa, slow down. There is no "ban" on incandescent light bulbs. It's just that the government passed new efficiency standards, and incandescents don't meet them.<br />
C: Oh, that's clever! I should try that some time: "See, I'm not breaking up with you! I'm just raising my standards to the point where you no longer qualify."<br />
<br />
<hr><br />
[Story: a pedophile was caught because he took pictures of his acts and tried to blur out the victims' faces, but police analysts were able to unblur them.]<br />
<br />
A: Hah! What an amateur! Everyone knows you have to do a true Gaussian blur to destroy the information content of the picture!<br />
B: Yeah, or entropize it by blacking out the whole face.<br />
C: Right. Or, you know, you could just ... not molest children.<br />
<br />
(IIRC, C was heavily voted down and criticized for assuming guilt.)<br />
<br />
<hr><br />
[Story: police used "big data" analytics techniques and discovered that most robberies occur on paydays near check-cashing places, which allowed them to ramp up arrests.]<br />
<br />
A: I don't know, this seems kind of big-brothery...<br />
B: Not at all! This is the kind of police work we should applaud! Working only off publicly available, non-private data, they found real, actionable correlations. It wasn't just some bigoted cop working off his gut: "Oh, this must be where the thugs go ..." No, they based it on real data. What's more, it let them avoid the trap of guessing the wrong paydays, which can actually vary! Some people get paid weekly, some biweekly, some of the 1st and 15th. For example, I get paid on the 7th and 21st.<br />
C: So, uh ... where do you cash your checks, by chance?Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com3tag:blogger.com,1999:blog-1145198247762305957.post-42914653938485802992015-12-30T11:50:00.002-06:002015-12-30T11:51:28.854-06:00What every programmer *really* should know about ...So there's a common kind of article that comes up a lot, and annoys me. It's the "What every programmer should know" or "needs to know" about this or that. About time, or names, or memory, or <a href="http://codecapsule.com/2014/02/12/coding-for-ssds-part-6-a-summary-what-every-programmer-should-know-about-solid-state-drives/">solid state drives</a> or floating point arithmetic, or file systems, or databases.<br />
<br />
<a href="https://hn.algolia.com/?query=what%20programmer%20should%20know%20about&sort=byPopularity&prefix&page=0&dateRange=all&type=story">Here's</a> a good sampling on Hacker News of the kinds of things someone out there is insisting you absolutely have to know before you can knock out "Hello, World", or otherwise meet some higher standard for being a "real programmer".<br />
<br />
I hate these articles.<br />
<br />
For any one of them, you can find about 99% who don't already know something from the article, and yet they manage to somehow be productive enough that someone is paying to produce a product of value. To me, they come off as, "I want to be important, so I want to make it seem like my esoteric knowledge is much more important than it really is." If taken seriously, these articles would heavily skew your priorities.<br />
<br />
So here's my standard for when you're justified in saying "what every programmer needs to know about X":<br />
<br />
Think about the tenacious, eager kid from your neighborhood. He picks up on things really quickly. Once he learned to program, he was showing off all kinds of impressive projects. But he also makes a lot of rookie mistakes that don't necessarily affect the output but would be a nightmare if he tried to scale up the app or extend its functionality. Things you, in your experience, with your hard-earned theoretical knowledge, would never do. When you explain it to him, he understands what you mean quickly enough, and (just barely) patches that up in future versions.<br />
<br />
What are those shortcomings? What are those insights, that this kid is lacking? What of this kid's mistakes would you want to give advice about to head off as soon as possible? <i>That's</i> what every programmer needs to know.<br />
<br />
The rest? Well, it can wait until your problem needs it.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com3tag:blogger.com,1999:blog-1145198247762305957.post-24913526931701206322015-12-06T22:15:00.001-06:002015-12-06T22:15:45.932-06:00The Scylla-Charybdis Heuristic: if more X is good, why not infinite X?(Charybdis = <a href="https://en.wikipedia.org/wiki/Charybdis">care-ib-dis</a>)<br />
<br />
A Scott Alexander <a href="http://slatestarcodex.com/2015/11/03/what-developmental-milestones-are-you-missing/">post</a> really resonated with me, when he talked about one "development milestone" (#4): the ability to understand and speak in terms of tradeoffs, rather than stubbornly try to insist that there are no downsides to your preferred course of action. Such "development milestones" indicate a certain maturity of one's thought process, and greatly change how you discuss ideas.<br />
<br />
When a Hacker News <a href="https://news.ycombinator.com/item?id=10504733">discussed</a> Alexander's post, I remembered that I had since started checking for this milestone whenever I engaged with someone's advocacy. I named my spot-check the "Scylla-Charybdis Heuristic", from the <a href="https://en.wikipedia.org/wiki/Between_Scylla_and_Charybdis">metaphor</a> of having to steer between two dangers, either of which have their downsides. There are several ways to phrase the core idea (beyond the one in the title):<br />
<br />
<blockquote>Any model that implies X is too low should also be capable of detecting when X is too high.</blockquote><br />
Or, from the metaphor,<br />
<br />
<blockquote>Don't steer further from Scylla unless you know where -- and how bad -- Charybdis is.</blockquote><br />
It is, in my opinion, a remarkably powerful challenge to whether you are thinking about an issue correctly: are you modeling the downsides of this course of action? Would you be capable of noticing them? Does your worldview have a method for weighing advantages against downsides? (Note: that's not just utilitarian cost/benefit analysis ups and downs, but relative moral weight of doing one bad thing vs another.) And it neatly extends to any issue:<br />
<br />
- If raising the minimum wage to $15/hour is a good idea, why not $100?<br />
<br />
- If lifting restrictions on immigration is good, why not allow the entire Chinese army to cross the border?<br />
<br />
- If there's nothing wrong with ever-increased punishments for convicts, why not the death penalty for everything?<br />
<br />
One indicator that you have not reached the "tradeoff milestone" is that you will focus on the absurdity of the counter-proposal, or how you didn't advocate for it: "Hey, no one's advocating that." "That's not what we're talking about." "Well, that just seems so extreme." (Extra penalty points for calling such a challenge a "straw man".)<br />
<br />
On the other hand, if you have reached this milestone, then your response will look more like, "Well, any time you increase X, you also end up increasing Y. That Y has the effect/implication of Z. With enough Z, the supposed benefits of X disappear. I advocate that X be moved to 3.6 because it's enough to help with Q, but not so much that it forces the effects of Z." (I emphasize again that this method does not assume a material, utilitarian, "tally up the costs" approach; all of those "effects" can include "violates moral code" type effects that don't directly correspond to a material cost.)<br />
<br />
I've been surprised to catch some very intelligent, respected speakers failing this on their core <br />
<br />
What about you? Do you make it a habit to identify the tradeoffs of the decisions you make? Do you recognize the costs and downsides of the policies you advocate? Do you have a mechanism for weighing the ups and downs?Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com5tag:blogger.com,1999:blog-1145198247762305957.post-90471738094633312112014-12-10T12:27:00.001-06:002014-12-10T12:28:51.851-06:00Bitcoin mining pools attacking each other for profit?So after posting our <a href="http://understandingbitcoin.us">new Bitcoin book</a>, my buddy <a href="https://twitter.com/ofnumbers/">Tim Swanson</a> alerted me to the problem of <a href="http://hackingdistributed.com/2014/12/03/the-miners-dilemma/">Bitcoin mining pools attacking each other.</a><br />
<br />
Reminds me of the recent <a href="http://blog.tyrannyofthemouse.com/2013/11/i-explain-tax-interaction-effects.html">Tax Interaction Effect</a> post, where I had to unearth the core of a counterintuitive result: in this case, the claim that <i>not redeeming</i> some of your solutions can increase your return.<br />
<br />
I don't think I'm entirely there, but I at least have an analogy to understand the mechanics of the attack.<br />
<br />
<b>Bitcoin mining model:</b>As you might know, mining is like buying a bunch of (positive sum) lottery tickets. A fixed reward is given out every hour, then divided equally among the winning tickets. Some people join into pools, where they buy tickets with the proviso that *if* it's a winning ticket, they share the winnings equally among all the tickets in their pool.<br />
<br />
<b>The attack:</b> You use some of your money to buy tickets for someone else's pool (call it the "attacked pool", but hide and destroy the winning tickets for that pool.<br />
<br />
<b>The effect:</b> There are fewer total wins per period. Each (non-destroyed) ticket gets a larger fraction of the hourly reward. The attacked pool gets a smaller fraction the reward.<br />
<br />
<b>My response/confusion:</b> This increases the return to <i>all</i> winning tickets, not just those of the attacking pool, so the attacking pool effectively subsidizes all the others, and dilutes the value of its own tickets across the set of all players.<br />
<br />
But maybe I'm missing something here.<br />
Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com2tag:blogger.com,1999:blog-1145198247762305957.post-12948262996883665012014-12-09T19:43:00.002-06:002014-12-09T19:43:28.412-06:00Our new Bitcoin eBook is up!Phew, been a while, eh? Well, Bob Murphy and I have a new free eBook up about the economics and mechanics of Bitcoin! Check the <a href="http://understandingbitcoin.us/">site for it</a>, or, if you're too lazy, just go straight to the <a href="http://understandingbitcoin.us/ub.pdf">book itself</a>.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com1tag:blogger.com,1999:blog-1145198247762305957.post-36550841459142269442014-03-16T17:28:00.000-05:002014-03-16T17:28:15.469-05:00Tax interaction effects, and the libertarian rejection of user feesPhew! Been a while, hasn't it?<br />
<br />
I want to come back to the <a href="http://blog.tyrannyofthemouse.com/2013/11/i-explain-tax-interaction-effects.html">tax interaction effect (TIE) issue</a> from previous posts, and go over what I think has been bothering me about the TIE-based argument against the carbon tax shift.<br />
<br />
So, a high-speed review of why a <a href="http://consultingbyrpm.com/blog/2013/10/ross-mckitrick-explains-why-there-probably-is-not-a-double-dividend-with-carbon-tax.html">carbon tax shift (CTS) is inefficient</a>. The CTS, remember, involves a revenue-neutral reduction of taxes on capital (including land) and labor, replaced by a tax on carbon emissions -- specifically, those fuels that, when used, release carbon dioxide, in proportion to how much CO2 they release per unit.<br />
<br />
<h3>Review of the argument </h3><br />
And why could it be inefficient? Well, the harm of a tax increases faster than its rate. To have a revenue-neutral CTS, you have to "focus" the tax -- i.e. raise the same revenue from a smaller class of goods. This necessarily means a higher tax rate on the "focused" goods, and therefore higher induced inefficiencies (compared to the broader tax). When you further note that these taxes will, in effect, "stack on" to the existing labor and capital taxes, then the inefficiencies are even higher -- that's the TIE -- and could even swamp the environmental benefit from the emissions reduction."<br />
<br />
<b>But hold on.</b> Those very same steps are a case against <i>any</i> correspondence between "who uses" and "who pays", whether or not the payment is a tax! That's because you can always point out how "concentrating costs" leads to disproportionate inefficiencies, even and especially for textbook "private goods".<br />
<br />
That is, you could likewise say, "if people have to -- gasp! -- pay for their own cell phones, at $300/each, then that scares away all the people who can't pay $300 (after paying labor taxes, remember!), so you can an efficiency loss there. Plus, anyone who can steal the phone has a $300 incentive too, so people invest in ways to steal them, and you have to pay for countermeasures. Those go up quickly with the price of the good.<br />
<br />
<b>"Therefore, the government should just tax everyone to cover the cost, and then hand out the cell phones for free."</b><br />
<br />
<h3>Wait, that doesn't sound right ...</h3><br />
What's wrong with that argument? Well, a lot. So much that you probably already know the answer. It's for the very same reasons that many advocate user fees for any good that's excludable. Generally, whoever benefits should be the one to pay. (<a href="http://en.wikipedia.org/wiki/Cuius_regio,_eius_religio"><i>"Cuius lubido, eius sumptum."</i></a> -- "Whose desire, his expense.")<br />
<br />
As with those reasons in favor of user fees, you can make the exact same argument regarding the purported inefficiency of a CTS: <br />
<br />
"Yes, you get inefficiencies every time you concentrate costs like that. And yes, they disproportionately stack with whatever taxes you already had. But you <i>need</i> the fee structure to work that way in order to align incentives. The one who uses the scarce resource -- whether a cell phone, or atmospheric dumping capacity -- should be the one to pay for it, as this leads them to economize on the use of that resource, and if possible, route around it. That remains doubly so when exempting them from the expense would lead to further penalization of every other class of socially-useful activity."<br />
<br />
And that, I think, goes to the core of my original balking at the CTS/TIE argument.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com0tag:blogger.com,1999:blog-1145198247762305957.post-55228155064773625942013-11-23T18:39:00.000-06:002013-11-23T18:39:01.881-06:00Liberty vs efficiency: The real conflictLiberty: Being free of constraints<br />
<br />
Efficiency: Raising the state of the world as high as possible on everyone's preference ranking (or some aggregate measure thereof)<br />
<br />
You might have heard of Amartya Sen's <a href="http://en.wikipedia.org/wiki/Liberal_paradox">Liberal paradox</a>, which purports to show that the two necessarily conflict. Of course, as I <a href="http://blog.tyrannyofthemouse.com/2010/10/setting-arrows-impossibility-theorem.html">said a while back</a>, it does no such thing; it only shows a problem with preventing people from <i>waiving</i> their liberties when they find it preferable to do so.<br />
<br />
However, there is a real sense in which those two conflict, and it becomes most apparent in discussions of taxation, and how to make it better.<br />
<br />
The conventional economist's view is that "The ideal tax system is the one that hurts efficiency the least."<br />
<br />
But there's another view, exemplified by the <a href="http://mises.org/daily/5915/">Murphy article</a> that I linked in my last post: "The ideal tax system is the one that's easiest to opt out of."<br />
<br />
Naturally, these really do conflict. Why? Because generally speaking, if you want to levy a tax that <i>merely</i> transfers purchasing power to the government without also forcing people to bear other hardships, you have to do it by taxing goods with inelastic demand, like energy, as people will not respond to the tax by buying less of the good, which would indicate a reduction in efficiency.<br />
<br />
But the harder a tax is to avoid, the harder it is to "opt-out" of!<br />
<br />
So if you think it's good for people to be able to legally reduce government revenues by abstaining from a product at relatively little cost to themselves, then "economically efficient taxes" are no longer an unvarnished good, as they come at the direct expense of the goal of making it easier for people to change their behavior in a way that routes around taxation.<br />
<br />
This, I think, is the true conflict between efficiency and liberty, as it doesn't hinge on confusing rights and obligations.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com2tag:blogger.com,1999:blog-1145198247762305957.post-30364507362073263662013-11-09T15:52:00.000-06:002013-11-09T15:54:45.661-06:00 I explain tax interaction effects (because I think the experts can't)So it turns out there's a <a href="http://www.youtube.com/watch?v=yxAztNFqxEk">serious argument</a> (HT and text summary: <a href="http://consultingbyrpm.com/blog/2013/10/ross-mckitrick-explains-why-there-probably-is-not-a-double-dividend-with-carbon-tax.html">Bob Murphy</a>) that a "green tax shift" may be welfare-<i>worsening</i> rather than welfare-improving. (The green tax shift is where you cut taxes on labor and capital while raising them on environmental "bads" like CO2 emission.)<br />
<br />
Huh? How can a tax shift off of bads and onto goods be welfare worsening? It seems the argument is somewhat subtle; even Bob Murphy <a href="http://consultingbyrpm.com/blog/2013/10/ross-mckitrick-explains-why-there-probably-is-not-a-double-dividend-with-carbon-tax.html#comment-76546">dismisses clarification requests</a> in the comments, pleading that "it’s hard to point to 'what’s driving the result' except to say, 'Adding the carbon tax drove the result.'"<br />
<br />
Well, it's actually not that hard, but the standard expositions don't make it explicit. After reading <a href="http://www.econlib.org/library/Columns/y2012/Murphycarbon.html">another</a> of Murphy's articles, it finally <a href="http://consultingbyrpm.com/blog/2013/10/ross-mckitrick-explains-why-there-probably-is-not-a-double-dividend-with-carbon-tax.html#comment-76599">clicked</a> for me, although the better explanations <i>still</i> hid the true mechanism in unstated assumptions. Here's how I <a href="http://consultingbyrpm.com/blog/2013/10/ross-mckitrick-explains-why-there-probably-is-not-a-double-dividend-with-carbon-tax.html#comment-76599">explained it in the comments</a> (cleaned up a bit and sourced).<br />
****<br />
I think I have an explanation that conveys the intuition.<br />
<br />
Insight 1: the harm of a tax is more-than-proportional to its magnitude. (This is the assumption that the writing on this seems to assume and which I wish was made explicit here and in your article.) Mankiw gives the rule of thumb that <a href="http://gregmankiw.blogspot.com/2008/10/average-effective-marginal-tax-rates.html">the deadweight loss of a tax increases with the square of the tax rate</a>. Thus why you want to raise a given amount of revenue from as “broad a base” as possible -- to lower the rate each tax has to be.<br />
<br />
Insight 2 (most important): Because of the above, each increase in tax above the Pigovian level is more harmful than the same increase from zero.<br />
<br />
Insight 3: Taxes on anything chase back to their original land/labor/capital factors. So a carbon tax amounts to a tax on land, labor, and capital, divided up per their relative supply/demand curve elasticities (slopes).<br />
<br />
Given the above, the intuition becomes a lot clearer: a tax on carbon is like an income tax (with different levels for different kinds of income). Levied at the Pigovian rate, it merely cancels out the carbon harms. But if you have an additional (direct) income tax, you get a disproportionate harm for each (potentially) taxed dollar above the Pigovian level (compare to taxing from the first dollar) — <b>*that* is the tax interaction effect.</b><br />
<br />
Furthermore, since the “green tax trade” tries to raise the same revenue on a smaller base (i.e. only those income sources touching carbon), the tax rates have to be much higher than they would be if they were on all income. This then causes major welfare-harming changes in behavior, far out of proportion to the assumed harms from carbon.<br />
****<br />
Problem solved, right?<br />
<br />
Well, no; Bob insists that Insight 1 is irrelevant to the argument. But I don't see how this can be; you can only get the bad "tax interaction effects" if the tax's harms are more-than-proportional to ("superlinear in") the tax rate.<br />
<br />
If it's merely proportional, the taxes don't "interact" at all -- raising taxes by 1 percentage point (on any kind of income) does just as much additional harm, regardless of whether it's on top of a 6% existing tax, or a zero. But when it's <i>more</i> than proportional, then that extra point of tax is (badly) "interacting" with whatever other taxes got it to that level. This is the key insight: that having income taxes in addition to the (implicit income tax resulting from a) carbon tax means those taxes are doing more harm than they otherwise would.<br />
<br />
Likewise, if the harm (deadweight loss) of a tax were <i>less</i> than proportional to (sublinear in) the rate, then they would interact in the <i>opposite</i> way. It would make sense to have as few distinct taxes as possible, on a small a base as possible, with as high a rate as possible -- because in that case, each additional increase in the tax rate hurts less than the previous. (Obviously, we don't live in that world!)<br />
<br />
I note, with some irony, that this point ultimately reduces to the reasoning behind standard mainstream economist's tax advice to "lower the rates, broaden the base", a mentality Bob actually criticized in <a href="http://mises.org/daily/5915/">another context</a>...Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com6tag:blogger.com,1999:blog-1145198247762305957.post-67763015115642429872013-11-07T12:56:00.002-06:002013-11-07T12:56:34.178-06:00No politician has ever liedBecause gosh, you'd <a href="http://www.thebigquestions.com/2013/11/07/obama-did-not-lie/">have to be an idiot to believe them</a> in the first place, says Steve Landsburg.Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com0tag:blogger.com,1999:blog-1145198247762305957.post-16300123549061997562013-07-11T00:17:00.003-05:002013-07-11T00:17:46.464-05:00My discovery of "semantic security"One interesting thing I forgot to mention in the previous <a href="http://blog.tyrannyofthemouse.com/2013/05/i-added-your-numbers-and-i-have-no-idea.html">post about homomorphic encryption</a>: the concept of <a href="http://en.wikipedia.org/wiki/Semantic_security">semantic security</a>.<br />
<br />
It was actually a major stumbling block for me. When I got to the passage that mentions the concept, the author casually remarks that "semantic security is an expected feature of public key cryptosystems", and then defines the term as follows: a system is semantically secure if, given two plaintexts and the ciphertext of one of them, an attacker cannot do better than chance in guessing which plaintext goes with that ciphertext.<br />
<br />
That didn't make sense because I had always assumed that the defining feature of public key cryptography was that the attacker is permitted unlimited chosen-plaintext attacks, which -- I thought -- means that the attacker always gets to know what ciphertext goes with any plaintext. So how can you make it so that the attacker can -- as required for public key encryption -- produce valid ciphertexts from arbitrary plaintext, and yet still have a semantically secure cryptosystem? Couldn't the attacker just encrypt both plaintexts to figure out which one corresponds to the given ciphertext?<br />
<br />
What I missed was that you can use a one-to-many cipher: that is, the same plaintext corresponds to many ciphertexts. What's more, it's actually trivial to convert a plain-vanilla one-to-one public key cipher into a one-to-many semantically secure version. Here's how: just before applying the encryption step, generate a random number (a "nonce") and append it to the plaintext, with the proviso that the recipient with look for it in that position and strip it off after decryption.<br />
<br />
This way, in order for an attacker to try to guess the plaintext in the game above, it's no longer enough for them to simply encrypt both plaintexts: a random number was inserted in the process. This means that in order to find a match between a plaintext and a ciphertext, the attacker must encrypt each plaintext with <i>every possible nonce</i>, which requires resources that increase exponentially with the size of the nonce used. (That is, an n-bit nonce can have 2^n possible values.)<br />
<br />
The more you know (tm).Silas Bartahttp://www.blogger.com/profile/09480427306873460464noreply@blogger.com6