guest - flak

medium rare

Is it crazy that a Medium post about javascript bloat would have itself have megabytes of javascript and stylesheets? I wouldn’t know, since I didn’t see it. I have a little proxy like service running that rewrites its HTML. This particular service was an experiment to replace some python code with go, to evaluate suitability for future hacks.

I’ve been using the python lxml library for HTML parsing for ages. Seems to work pretty well. There’s actually a bunch of little one off scripts that share a similar skeleton, which is modified as needed. After all, the best code isn’t reusable, it’s reeditable. A little while ago that turned into a script to download Medium posts after I read them and save the important parts, so that sometime later when I want to read about the Riemann Hypothesis, it’s all still there in a place I can find it.

Continue reading medium rare...

Posted 2017-02-13 14:13:00 by tedu Updated: 2017-02-18 20:10:56
Tagged: go programming web


Hello there, inquisitive friend! I’m pleased to announce the newest Links As A Service offering. It’s called inks which is like links, but without the L for loser. Basically Reddit or Hacker News, but without the disagreeable trolls and military industrial complex shills downvoting everything to hide the truth.

Posted 2016-11-06 18:40:26 by tedu Updated: 2016-11-06 18:40:26
Tagged: project web

cloudflare and rss

Let’s say somebody has a blog that I’d like to read. Subscribe to even. Let’s say they have an RSS link on their page. This should be easy.

Now let’s say the blog in question is hosted/proxied/whatever by Cloudflare. Uh oh.

Just reading the blog in my browser is now somewhat hampered because Cloudflare thinks I’m some sort of cyberterrorist and requires my browser to run a javascript anti-turing test. But eventually the blog loads, I read it, click the RSS link to subscribe, see that it is in fact XML rendered in my browser, and copy the link.

I paste the link into my RSS reader, optimistically hoping to see new links arrive. But they never do. Check the logs. Seems I’m getting 503 server errors, which is Cloudflare’s way of saying, “It’s not us; it’s you. And fuck off.”

Apparently my feed fetcher is also a cyberterrorist. It’s also written in python and can’t solve browser detecting riddles because it doesn’t include a javascript engine because OMG why would fetching an RSS feed require javascript?

Now I’m somewhat less inclined to read said blog, but hey, at least the internet is being kept fast and secure from miscreants like me.

Posted 2016-09-16 04:59:40 by tedu Updated: 2016-10-05 17:59:11
Tagged: rants web

tweet compression

Sometimes you’ve got something really important to tweet, but it doesn’t quite fit in 140 characters. There’s several techniques that can help in this situation.

One option is to use another platform that allows longer posts, but like I said, this is really important. Twitter or bust. Or write the post on a napkin, then upload a picture of it, but let’s pretend we have an irrational preference for textual information conveyed as text.

English text has a lot of redundancy. Certain digraphs are particularly common. In fact, many digraphs are actually descended from single letters. We can shave some characters from our tweets by winding back the clock.

If instead of writing “This or that” we write ”Þis or þat” using thorns, thankfully preserved in unicode by our viking friends, that represents a savings of 16%. It’s the same number of bytes, but I don’t make the absurd rules.

Continue reading tweet compression...

Posted 2016-08-27 19:00:59 by tedu Updated: 2016-09-14 14:37:00
Tagged: javascript language web

Lo and Behold

Werner Herzog reflects on the reveries of the connected world. There’s a lot of short sequences here, but not much tying it together.

We start in the building with the ugly hallways at UCLA where the first internet connection was established. The first message transmitted was supposed to be “login”, but the machine crashed after “lo”. Lo and behold.

The inventor of cut and paste doesn’t like what’s been done to it.

The law of large numbers means that the bigger the internet gets, the more efficient it becomes. Everybody talking to everybody averages out. I’m not sure how this theoretical result squares with the reality that Netflix is 33% of traffic.

Continue reading Lo and Behold...

Posted 2016-08-25 01:00:09 by tedu Updated: 2016-08-25 18:04:23
Tagged: moviereview network web

rss table manners

I provide an RSS feed for flak. I also wrote a simplistic RSS feed reader for myself. The design of the latter was influenced by observing the behavior of existing readers.

There’s a small wave of fetchers that appear every five and ten minutes, converging with larger waves every fifteen minutes. These coalesce with a tidal wave at the top of every hour. My log file shows a whole lot of quiet interspersed with feeding frenzies at regular intervals.

This isn’t a problem, per se, because the total number of feeders is low, and the feed itself is very lightweight. But it’s easy to imagine a more popular blog with more content requiring an outsize investment in capacity to handle such an uneven request distribution.

What can a reader do to avoid such rude behavior? Check feeds at irregular times. For me, this was implemented as a check deadline for each feed. Each time the feed is checked, the deadline is incremented by a random amount between two and four hours. (One to two would work great, too. I’ve fluctuated a bit.) This means that not only is my fetcher not synced with other fetchers, but it’s not possible for it to even accidentally fall into lock step.

If everyone did things this way, that’s all that would be needed. But in a world populated with lock step feeders, there’s one more wrinkle. The fetch process is initiated by cron every five minutes, but the very first thing it does is sleep a random amount between one and three minutes before checking for expired deadlines, ensuring that we never hit a server during a hot minute.

I do this mostly because being polite to servers is the right thing to do, but clients benefit from being nice too. Requests to an idle server are more likely to succeed and faster. If multiple clients are sharing a link (or proxy), they can suffer the same kinds of congestion that busy servers do.

One can imagine that RSS feeds are not the only problem domain which benefits by decoupling a regular activity from a fixed time.

Posted 2016-07-27 18:00:56 by tedu Updated: 2016-07-27 18:00:56
Tagged: software web

the future is arriving too fast

Because I am old, sometimes instead of watching new original content, I want to watch old preexisting content which is not available on Netflix or any other streaming service. Fortunately, there is a solution. Netflix also has a service which will mail me plastic circles that I can watch by putting them in my plastic circle player. I can manage the queue of such circles by using my browser. Ah, the wonders of technology.

Also because I am old, sometimes I go talk with other old people, in person, at bars and such. Mostly we reminisce about the old days, when we had to seduce people with words instead of pictures of our junk. But sometimes the conversation turns to entertainment, such as movies. Somebody might claim that Jupiter Ascending by the Wachowskis is the spiritual successor to The Matrix. This is obviously a claim that needs to be seen to be believed. (Though I recommend neither seeing nor believing.)

Continue reading the future is arriving too fast...

Posted 2016-04-06 18:35:40 by tedu Updated: 2016-04-06 18:35:40
Tagged: business rants web

moderation in moderation

William “the Jar” Mason is a semi famous programmer. Mostly retired, but his website still has some classic postings from early days working on essential software tools like vi and lynx. Unzealous Association is a link aggregator popular among people who like to read Mason’s articles.

The trouble begins one day when wjm decides that UA sends too much traffic his way. Like a denial of service. And so wjm responds by redirecting anyone with a referer of UA to a picture of a roast ham. (This is probably an overreaction. It’s not really the UA users at fault, but the many aggressively stupid bots that scrape all linked sites. But it has the desired effect of keeping links to wjm’s site off the front page.)

This action is not without collateral damage. It’s not just that headline links disappear, but also less trafficked links in comments are affected. This then incites an unhelpful mini thread on UA about how the internet works.

The UA response is to autokill any comments linking to wjm. The comment is hidden from most users, but remains visible to the author without any indication of what went wrong. (Also Known as hellbanning, the nuclear option of troll containment.)

There are a couple other ways this could have played out. Possibly, if the UA software can detect wjm links in order to kill them, it could also skip adding the <a> tags. Users who cut and paste the link don’t have referer headers. Problem solved. Another option might be to simply ride it out and see if the complaint threads dissipate. Maybe wjm will even change his mind some day.

Unfortunately, when all you have is a trollhammer, all you see are trolls.

Posted 2015-12-21 15:35:00 by tedu Updated: 2015-12-21 15:35:00
Tagged: rants web

pinboard tips for web design

A few funnies sprinkled with a bit of insight and disappointment. And regret.

It’s 2015. Your team has to wake up determined and put in one hell of a work week to get web pages to render slowly. And yet so many succeed. tweet.

My modest proposal: your website should not exceed in file size the major works of Russian literature. Anna Karenina, for example, is 1.8 MB. tweet.

If your design team insists on including a lot of Javascript cruft and CSS resets, make them write it all out longhand with a quill pen. tweet.

Continue reading pinboard tips for web design...

Posted 2015-11-04 06:10:03 by tedu Updated: 2016-01-01 14:04:24
Tagged: quote rants web

bring your own customer service

Skip the middleman to save time and money by simply telling your customers exactly what you would have told your customer service team. Simple direct communications mean nothing gets lost in translation. Not even funtioning.

Best of all, if they screw up, it’s their own fault.

Posted 2015-10-23 17:42:54 by tedu Updated: 2015-10-23 17:44:28
Tagged: bugs web

a prettier web, not a thicker one

There’s been a lot of fuss recently about the state of the web. quirksmode got the party started by telling us to stop pushing the web forward. Enough, enough, there’s too much! From the other direction, The Verge points out it’s really only too much because Microsoft refuses to release IE for iPhone. Whatever. For the morbidly curious, two fairly long recaps are Stop blaming the web. Stop breaking the web. and What’s wrong with the web?

Mostly the focus has been on overwhelming cognitive load for developers and a worsening user experience for, uh, users. What about security? Or privacy? The things nobody cares about because they can’t be A/B tested. Let’s take a look at a few feature fuckups. Bear with me, I had to dig to find these examples, so some links could be as much as a month old.

Continue reading a prettier web, not a thicker one...

Posted 2015-08-13 17:05:33 by tedu Updated: 2015-08-13 17:05:33
Tagged: software thoughts web

on the detection of quantum insert

The NSA has a secret project that can redirect web browsers to sites containing more sophisticated exploits called QUANTUM INSERT. (Do I still need to say allegedly?) It works by injecting packets into the TCP stream, though overwriting the stream may be a more accurate description. Refer to Deep dive into QUANTUM INSERT for more details. At the end of that post, there’s links to some code that can help one detect QI attacks in the wild. As noted by Wired and Bruce Schneier, among dozens of others, now we can defend ourselves against this attack (well, at least detect it).

Continue reading on the detection of quantum insert...

Posted 2015-08-06 02:24:12 by tedu Updated: 2015-08-06 02:24:12
Tagged: project security software web

bad robot

The best part of running your own server is definitely reviewing the logs. There are a lot of silly people out there, and each and every one of them has written a program that would like to visit your server.

The fun comes from watching each bot, then trying to guess the nature of the bug.

Continue reading bad robot...

Posted 2015-08-04 11:34:08 by tedu Updated: 2015-08-04 11:34:08
Tagged: rants software web

rolling expired certs

This wasn’t the post I intended to write today, but then I noticed that the certificate for had expired, and repairing that became a prerequisite for getting anything else done. At the time, my first snarky thought upon discovering Firefox wouldn’t let me connect to my site anymore was “Oh, hurray, don’t I feel safe.” Then I went through the update nonsense and thought a bit more seriously about it.

My cert expired after a year because that seems to be the thing to do. I imagine there’s some nebulous threat model where somebody stole my server key and has been impersonating me for the past six months, but now they can’t. Although, if they stole the old key, they can probably steal the new key. I suppose we do this because revocation doesn’t work, but a six month half life is a long time to sit exposed.

Continue reading rolling expired certs...

Posted 2015-07-08 18:46:29 by tedu Updated: 2015-07-08 18:46:29
Tagged: rants security web

twitter spam problem

It’s still fashionable to explain why (random internet company) is going downhill, right? Here’s why Twitter sucks. They have a spam problem and they’re not doing anything about it.

I occasionally search twitter for OpenBSD. Unfortunately, it’s been taken over by ad bots. Is it necessary to do the realtime search? Often times, yes. Otherwise Twitter tends to keep showing me the same set of tweets from last week over and over.

Exhibit 1:

How can Twitter not detect bullshit accounts like this? You can try reporting them, which I have, but obviously that has had no effect.

For more giggles, exhibit 2:

When the name of the account even includes the word spam, surely that must be a hint?


Twitter does have a spam filter! After posting this tweet I was notified I was posting too much spam and my account was locked.

Hi Ted Unangst, Your account appears to have exhibited automated behavior that violates the Twitter Rules.

Nothing to worry about. Twitter’s spam team is on the ball.

Posted 2015-02-01 02:44:23 by tedu Updated: 2015-02-11 00:48:04
Tagged: rants web

full screen clippy

Every time I watch a full screen video, Chrome feels the need to tell me that YouTube is now full screen. Oh really? I already knew that. How did I know that? Because I just clicked the fucking full screen button.

Why is Chrome compelled to tell me something I already know? Oh, right, new users. Maybe somebody clicked the button by accident. So that justifies, what? One, two warnings? Three? Surely not three hundred. This isn’t accommodating; this is aggravating.

Let’s say I decide to relax and spend an hour watching music videos (it’s cold outside!). In that span, I will be faced with 20 reminders that yes, once again, YouTube is full screen. (There’s no search in full screen mode, and while I may enjoy one Cars video, I don’t need to watch the entire set.) By now even a lobotomized lab rat will have figured out that, just like the previous 19 times this happened, I can press Esc to exit. But not me. Chrome isn’t quite sure I’m smart enough to remember which key to press. Thanks for the vote of confidence, guys.

Now assuming I do have the memory of a gold fish, how does the reminder at the beginning of the video help? By the time I’ve watched the video, I’ve surely forgotten the annoying popup that was blocking my view and which I didn’t read. Once I do find the Esc key, however, be sure to remind me about it again seven seconds later when I play the next video.

It’s the Return of Clippy. I noticed you’re watching a video full screen. Does that mean you want to see the whole thing? Is this dialog interfering with your viewing? Would you like this dialog to go away and never come back? Don’t worry, I’ll be right over here in case you need me.

Ironically, Internet Explorer does get this right. Asks me once, the first time, if I want to switch to full screen. Yes. Never a peep after that.

Posted 2015-01-29 06:04:18 by tedu Updated: 2015-01-29 07:53:52
Tagged: rants software web

where did the cookies go?

Not always, but more frequently than never, I manage Firefox’s cookies by hand. Seeing what’s set, clearing out some I don’t like. Recently I discovered the button to do so in the Preferences dialog had disappeared from the Privacy tab.

Where did it go? It’s hiding under the history section. You have to change Firefox will: “Remember history” to “Use custom settings for history” and then the “Show Cookies...” button reappears. Because that totally makes sense. Just looking at cookies clearly requires that I also change to custom history settings.

Posted 2014-12-15 06:01:21 by tedu Updated: 2014-12-15 06:01:21
Tagged: rants software web

timing attacks vs hash tables

First, start with there are no good constant-time data structures. After reading the HN thread, I wanted to see if the attack was truly viable. Can we recovery a JSESSIONID? My previous efforts attacking Lua took a slightly different tack.

To start, we need a vulnerable server. Worst case would be a simple hash table with chaining, such as one might build with the BSD queue macros. We’ll use a very simple hash function to make controlling the bucket simpler. And, of course, strcmp.

I’m exploiting some of my own knowledge attacking this server. For instance, I know there’s only four buckets, and I know the hash function, and I know the correct token values. The purpose wasn’t to build a fully weaponized attack, just to identify what timing irregularities may exist.

Continue reading timing attacks vs hash tables...

Posted 2014-12-03 08:51:49 by tedu Updated: 2015-06-16 18:43:31
Tagged: c programming security web

easy mobile passwords

Matthew Green asked for a password generator that’s easy to enter on a phone.

Here’s one solution that works for the iPhone keyboard. To make it easy to type with your thumbs, it alternates sides of the keyboard for each letter. Sometimes it throws in a shift. Sometimes it throws in a symbol, but only one from the right side since it requires before and after left taps to get there. In practice, it appears to generate passwords that I can or could at least learn to type fairly quickly.

Continue reading easy mobile passwords...

Posted 2014-09-01 23:00:27 by tedu Updated: 2014-11-30 22:18:32
Tagged: gadget lua programming security web