Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway

[ music | Depeche Mode – Route 66 ]

I was in Houston a few weeks ago, and I had many hundreds of thousands of multi-megabyte files I need to shove through a 20Mbit upstream connection to Azure. Strictly speaking the problem here wasn’t how long it would take to send the file data, that could be done in two-ish days on that pipe. However, 500GB of data takes a lot more time than 55 hours when it’s spread over hundreds of thousands of files being written to a Windows server on NTFS. This transfer was looking to take more than a week to complete.

To keep track of all your files on a disk, the file system has features to keep “catalogs” of everything. Just like the card catalog in a library tells you where every book is, there are similar features in a filesystem to keep track of where every part of a file is, where the empty space is, etc. So when you create a file, the file system has to keep track of every single action, recording where every part of every file is as it happens. There can be a lot of time overhead as this data is created, recorded, and verified, much mroe so than just dumping bytes to the disk. When you’re creating hundreds of thousands of files across a slow link, every single millisecond of extra delay per file adds up to about nine minutes across the overall transfer.

The solution? My plane trip home. We slapped an NVMe SSD into the VMware server, created a new disk on the virtual server, copied the files, and I took the NVMe drive home where I have a symmetrical gigabit internet connection (it’s 1 gig up and down, most cable internet connections ar emuch faster down than up). I popped the drive in on of my home VMware servers, attached it to a Windows machine, created the VPN link in the VM to Azure. Now I still had the problem of 500,000 files, however.

It still would have taken a couple days with file system overhead across a network link. But it only took a couple of hours to zip up all the files into a single giant ZIP file, which took 2 hours to upload, and then 3 hours to decompress it on the other end. Local file system access is always faster than network file system access, so zipping and unzipping the file was far, far faster than the network transfer would have been.

This is why almost 40 years ago Andrew Tannenbaum came up with the brilliant quote, “Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.

Comments off

Modifying a Dell 3020m micro PC for dual gigabit ethernet; adventures in Dremeling.

[ music | Lady Gaga – I Like It Rough ]

I’m lucky enough to have AT&T Fiber for gigabit internet, and while the rest of my network stack in the house is Cisco Meraki, I can’t afford an MX250 to use the whole pipe, or even an MX100 for 75%.

Used to use a 2011 Mac Mini as a pfSense-based router here. I had the thunderbolt to gigabit ethernet adapter, and it worked fine, but I wanted something else. No real reason, no good excuse, but I lied to myself and said a second gen Core i5 (2415M, doal core at 2.3GHz, Sandy Bridge) wasn’t “enough”. And I wanted something “less hacky” than a thunderbolt ethernet adapter. Also, it ran warm, I really did want something to run cooler.

Dell 3020m open case

Enter a Dell 3020M micro form factor PC. It had an i5-4950T, quad core, ran at 3.3GHz, Haswell, and had a gigE port and internally an m.2 slot for a wifi card. It was A/E keyed, not M keyed (I’ll complain about m.2 keys later) so trying to find an ethernet card to fit in that slot was a trip. And also, the card slot is only really compatible with 30mm length cards.

After several months of searching for an A/E keyed card, I finally found a reasonably priced card! However, it’s 60mm long, and the slot has absolutely no room past 30mm due to the audio ports and the front of the case/chassis.

So I eventually thought I’d found a solution, an A/E keyed m.2 extender! I didn’t look closely enough, and missed that it actually goes from A/E key on the slot side to M key on the card. Useless for this use case.

Enter the Dremel.

This is nothing a little soldering (well, desoldering) can’t cure! I had to remove the audio out and mic in jacks, so I desoldered them and mounted the card. The removed audio jacks allow the 60mm card to extend past the 30mm slot, and the 30mm screw hole keeps the card in place. Then I took a 6 inch CAT6 cable, cut off one of the jacks, fished it through a hole in the back of the chassis, and snapped on an EZ-SnapJack. I actually really like these as there’s no punchdown tool needed. I also removed the casing from the SSD, as it’s entirely useless, it doesn’t even make contact with the tiny card for heat dissipation. Also, losing the case and the drive caddy that it mounts into makes the interior a lot roomier. The cable comfortably goes over the SSD and out the back where the EZ-SnapJack is, and after these photos a little hot glue secured it to the rear of the case.

However, the card is too long to fit in the case, now. This is nothing a Dremel can’t cure! It ruins the stock exterior aesthetic I wanted, but a gaping hole in the front does improve airflow. The hole in the case also allows the 60mm card to extend past the boundaries of the 30mm slot.

It ain’t as pretty as I wanted, but I got everything else. Dual PCIe gigabit ethernet cards in a tiny form factor, and a faster machine for pfSense.

Post Script. On M.2 keying. IT’S STUPID AND UNNECESSARILY COMPLICATES THINGS. Look at this chart at Wikipedia, it’s insane. This is the same BS we have gone through in the IT world for decades. We needlessly add complexities to things that we think we need to “differentiate” when the differentiation serves no purpose. Serial ports were a mess, with 9 pin and 25 ports doing the exact same thing with the only difference being the physical connector. And different devices could have a male or female version of that connector, with no one real standard being followed. It was common to need a couple of different adapters on your cable to hook up two pieces of equipment. USB started with the A and B connector, which exploded into A and a whole world of B side connectors, finally coming to sense in the USB-C connector. Why does it matter which side of the cable goes where? It doesn’t. Whoever designed VGA cables understood that.

All you need in an M.2 slot is a single key to prevent M.2 SATA drives from being plugged in. That’s it. Everything else is PCIe and it doesn’t matter if wifi cards have a different notch from a USB card or a serial card, etc. “But what if it uses four lanes, or two?” Then it uses the lanes that are there, or it doesn’t. the connector doesn’t care, and the user doesn’t care. Keying should only be used to differentiate between electrically incompatible ports. Can you imagine if USB had notches for flash drives and ethernet adapters?

Comments off

LensCrafters, or, How Not To Make Glasses

[ music | Nine Inch Nails – I Do Not Want This ]

I went to LensCrafters on Friday, November 4. The eye exam went fine. The production of glasses went less well. I do not claim this goes for all LensCrafters locations, but I’m sure as hell done with this one. But first a little backstory.

I have 20/10 visual acuity. “Then why do you wear glasses?” Because your visual acuity is not necessarily the same as your uncorrected vision. Your visual acuity is the measure of your ability to discern objects according to a measurement standard. You uncorrected vision is measured by the same standard, but once you get glasses that correct your vision, you achieve your optimal visual acuity. Or at least you’re supposed to.

When my glasses are made properly, my 20/10 vision is attained quite nicely. The up side is I can read road signs from half a mile, and my phone with small text for privacy. The down side is that if the glasses aren’t made properly, I notice things most folks might not, and it drives me nuts. It can be so bothersome that I’m unable to wear the glasses at all. Three years ago I went to another spectacle vendor who were so sloppy that the eyeglasses they made were off by 14% from the designated prescription. This mistake happened three times in a row, the original pair and two remakes. I demanded a refund, and went to Walmart Optical who nailed it on the first try.

This year I have vision coverage with my insurance. So, given the discount, I decided to go to LensCrafters and get new glasses quickly. They don’t have their own optometrists, they partner with local doctors, leasing out part of the space so that it’s a one-stop-shop. Going in I explained to the optometrist the degree of my visual acuity and the need to go a little further with the exam. At a certain point the optometrist declared us finished, having achieved 20/20 vision. I said, “yes, but as I said I normally get back to 20/10 vision.” The doctor claimed not to know what I was talking about. “Those numbers don’t mean anything, you can see here your prescription is…” At this point I should have seen the writing on the wall. I literally explained what the whole 20/20 thing meant to an optometrist. After doing so, she stopped playing an idiot and said, “Well, I don’t normally test beyond 20/20.” I replied, “So I guess those numbers make sense after all.” So she hit a button on the testing machine, and we continued the test right back to 20/10 where I normally am. She was flabbergasted that I was right, that I actually had the visual acuity that I claimed! Amazing! A non optometrist knowing their own visual potential! It’s almost like I’ve been wearing glasses for 30 years!

Now I get to LensCrafters proper. I picked frames (I really liked them), sat down with a very nice optician, and with her help we went through the lens options, placed the order, and paid. I made sure again to mention the specificity that I need in the manufacture of my glasses. A little over an hour went by, and I was notified the glasses were ready. I went back, and tried on the glasses. Something was wrong. I looked around and except for the center, weird distortions and blurriness were all I saw. Even in the center something was wrong. The right eye was fine, but the left eye was blurry in the center. Then I looked to the left a bit with the left eye, and noticed things were perfectly clear. So if I looked straight ahead the right eye was clear, and if I looked to the left then the left eye was clear. Were I walleyed I’d be in fine shape, but alas I am not. My eyes both point in the same direction at the same time. Except through these glasses.

Before remaking them, I came back the next day to have the optometrist double check the prescription. It was a different doctor, but her exam agreed with the first. And the prescription itself was actually the same as my old ones in the left eye, and only very slightly different in the right eye, which is interesting because it’s the left eye that is giving us problems at this point in the story. So at this point I try to talk with the LensCrafters folks about the lens material; I had picked a high index material and I thought maybe that was causing some of the distortions I had noticed. I asked to go over again the lens options. I was told, “I don’t know what other lenses we have.” I said, “Well, let’s find out, we went over it when I was purchasing them yesterday.” Again I was rebuked, “Well, [the lens technician] has been doing this a long time, and he said just send it back for a remake.” Again I said, “Yes, that’s good, but I’d like to try to avoid a repeat of this issue.” “Well, he’s the expert so we’ll just send them back as he said.” It was Saturday, and they were busy, so I said that they could do it Monday when they’re less busy.

I went back today, Monday the 7th, and they were finishing them up at that very moment so I came back half an hour later. They were done. I tried them on, the only difference was the lack of a blurry stripe in the center of my left eye. The distortions and blurriness everywhere else hadn’t changed, not one iota. I explained this, and again said we need to go over the lens options. I again was told she didn’t know what they were. How can someone who rattles these off 8 times a day every day for years not have these options memorized? So again I said, “we went over this when I bought them, why can’t I just see a list of the options so we don’t keep repeating the same mistake? Why can I just get that question answered?” She looked at me and said, “I don’t know why you can’t just get that off the website.” I replied and said, “I didn’t know you HAD it on the website. I’ve asked for this half a dozen times and only now you tell me that? Besides, you showed me the list three days ago, why can’t I see it again now?” She said, “Well, I don’t even know if he’s going to want to remake them again,” and began to get up to go talk to the lens technician.

I asked her to stop, “Please, let’s get the question of the lenses sorted before we ask him because there’s no point in even asking if we don’t know what we’re asking for.” She replied, “well, I’m not sure there’s a point if he won’t remake them.” I said, “there isn’t a point if I can’t get a simple question answered.” She replied, “Maybe you should just get a refund.” I agreed, “if I can’t find out what my options are, then yes, I should.”

She processes the refund, but only for the regular glasses. I had ordered sunglasses as well, paid for out of my own pocked, eithout any insurance discount. I said, “This is only for the one pair.” She said, “yes, did you want to refund the other pair?” I said, “Since I can’t get an answer to a simple question to help ensure those don’t come out the same way as the first pair, then yes. I don’t want to keep going through this.” She still refuses to tell me my lens options again, and processes the refund.

I purchased two pairs of glasses, put the protection plan on both (where I get lenses remade should there be an accident for only $25 per incident, and she even recommended doing that before the year was up to ensure the lenses were in perfect shape), and because I had a problem, rather than discussing the problem with me, I was told to go away and take my money back.

So I’ve wasted six hours across four days only to be treated as though I hadn’t just spent a good deal of money on a product, and treated poorly because I dared try to ensure the product was functional for the purpose.

I work hard every day to do my job to the best of my ability. If someone asks me a question (much less eight times) I try to find an answer. Am I really so off base to demand the same thing when I drop hundreds of dollars on glasses?

Comments off

VMWare, BSODs, and video drivers

I had a customer who upgraded to an SSD in his MacBook Pro, and had heard about VMware Fusion. Not by name, but by feature, from a friend. “He can run his Windows right along side his Mac.” He had a Boot Camp installation and VMware Fusion helpfully can import that. However, after VMware Tools were installed, the VM would BSOD on boot with a SYSTEM_SERVICE_EXCEPTION, error code 0x0000003b. Oh joy.

Much testing later revealed that it seemed to be the VMware SVGA driver that was causing these issues, which made little sense as its one of the most tested parts of the VMware tools, being so critical. It turns out the client had a specific external video device that clearly VMware never tested with. In this case it was the Tritton SEE2 Xtreme USB to DVI adapter unit. While the unit wasn’t attached and hadn’t been used in ages, the drivers were still installed, and when these two drivers were initialized, we’d see that BSOD exception.

The solution was clearly to find and remove the driver for that video unit. Once that was done everything was aces. The moral is search very closely in the driver set of your guest OS if you’re having errors like this. I let VMware support know the details of this particular driver so they can test against it, and presumably devices like it.

Comments off

My views on Firefox OS

[ music | Styx – Fooling Yourself ]

Initially I blew off the Boot 2 Gecko initiative (now Firefox OS or something close to that) as unnecessary and pointless. I freely admit I was wrong. It’s actually a good idea, I like the idea, and I hope it works well. that said, I still think some of the goals are pipe dreams and pointless. I’m specifically replying to some of the things written in this blog post by Rob Hawkes (no relation to Guy), There is something magical about Firefox OS.

Feature phones. What? This is nearly 2013. Yes, there are probably literally a billion feature phones out there. There are reasons for that. They’re cheap, much more rugged than smartphones, last a week on a charge, and fulfill a basic need for communication via text and voice. They have very small screens with low resolution monochrome LCD displays. These are cheap in financial terms, power requirements,  and processing needs. There’s nothing Firefox can bring to this market that the cheap, small embedded OS can’t do, and no new benefits. To do anything smartphone related, you’ll need at least a better screen. Instantly you double the cost and reduce the disposability. Double? When you’re looking at hardware that costs $6 to $10 to produce and can be sold for under $20, yes, double.

Cheap smartphones. There are already cheap Android phones. A Firefox OS based phone MIGHT yield better web performance than Android on an underpowered phone, but screen space and computing power is getting so cheap that a 1GHz ARM powered phone will be under a hundred dollars by the time B2G hits 1.0. Now, if you make that type of phone perform better, that’s great, but let’s not pretend that’s really the goal here.

The goal of Firefox OS isn’t to compete with high-end devices, but to offer entry- to mid-level smartphones at feature phone prices. – Bonnie Cha

No it’s not. It’s to create an even more open competitor to Android. You can’t make a cheaper OS than android, because there’s no licensing cost to undercut. You can’t make the phones cheaper with software. Maybe Firefox OS makes underpowered hardware more usable, that doesn’t make the phone cheaper, just less crappy. Hardware is getting cheaper every six months, the problem is that they’re so cheap that it’s not worth the dev time to put Android on “feature” phone segment handsets.

The truth is Firefox OS might succeed where WebOS failed, and that is the exciting part. That’s why I did a 180 and began to like what FireFox OS could be. It’s more open than Android, has more dedicated backers than WebOS in both software and hardware partners, and based on proved tech rather than new tech. But give up on the hippie mantra of cheat smartphones for the masses. The masses will use yesterday’s tech. Yesterday’s hardware starts out as tomorrow’s, which Firefox OS might just make awesome for entirely different reasons.

Comments (9)

Kill the Meebo bar

[ music | Styx – Show Me The Way ]

I heard about the Meebo bar via this Gizmodo article about it, and how killed it within 48 hours. Well, the AJC here in Atlanta has begun using it now too. It’s crap, but I use Firefox which makes getting rid of it easy. Here are some simple steps to kill it on the AJC’s website.

  1. Find your Firefox Profile directory
  2. Open the folder labeled “Chrome”
  3. Create a new file named UserContent.css or edit the one already there if it exists. this file is just text, so use Notepad or another text editor
  4. Paste the following into the file:
    div#meebo.meebo-00 { display: none ! important; }
  5. Save the file, and restart Firefox.

That’s it. Now, other sites that use the Meebo bar can be cured just as easily, you’ll just need to know the proper identifier for the Meebo bar on that page. If you’re not technically inclined, find a technically inclined friend to do the following:

  1. Right click in a blank area of the toolbar, select “Inspect Element”
  2. Examine the DOM nodes in the breadcrumb bar at the bottom to find the parent element of the Meebo bar. Do this on and look for the identifier I used for that as listed above for an example.
  3. Copy the above code onto a new line, and edit the line in the above directions with the identifier for the site you’re concerned with.
  4. Save and restart Firefox.

Thanks to Gavin Sharp for pointing out the incredible usefulness of the new inspection tools, and everyone who contributed to the ifantastic new devtools in Firefox.

Comments (1)

You’ve got to fight for your right to party poker!

While online poker is the scourge of America, causing gay marriages and ruining our schools, Europe seems to have no problem with online poker.

For example, take a look at the Germans. In Germany they’re living life like normal despite the fact that one of the best online poker sites has a German specific area over at This site is the all-German language section of, which is a rather famous online poker company that you may have seen on shirts worn by players at the World Series of Poker. Millions of people play there and love it. Were I German, I’d play there too. So, if you’re German, or Swiss, Austrian, live on the eastern border of the Netherlands or Belgium, or you just speak German, go check out the best online poker site that’s all auf deutsch all the time.

Unless you live in America, then you should just hide.

Comments off

Via Dylan

I took a tour, I rode the wind,
I went to find, Went around that bend.

I saw brilliance, a thousand little lights,
they crashed to earth and lost their fights.

We looked around to see what we could see,
we saw the things we wished could be.

We dropped our guard, and let out a yell,
decided we’d all rather reign in Hell.

We were young a dumb, like lambs of the spring.
We drove off a cliff, we done a damn dumb thing.

We had it all in the palm of our hands,
we threw it away and ate the plans.

There were a million men with ideas of gold,
we’ll be lucky if fifty can grow old.

We tried drugs and sex and rock and roll,
and told ourselves we weren’t just foals.

We wandered west from the Island to the Beach,
To discover what the world had to teach.

But we couldn’t hear what the world had to say,
we yelled and screamed and got in the way.

Then we took up arms against the men of the old,
we wanted to heat up the long dark cold.

We decided to find out what if life was worth living,
to find what she had, to take what she’s giving.

We rolled up that hill, and fell off that line.
got in a fight, paid the damn fine.

After the spring, that flower done bloom.
That beautiful son, it filled up the room.

It lasted all day, and all night, and all year, and thousand more like it.

I don’t wanna look to deep or too long,
if push it too hard then the feeling is gone.

I have to collapse and roll around in the mud,
there’s no wolf at the door, this joint is a dud.

But there’s crazy shit happenin’ here in my head,
we’re all swimming in honey and lead.

There’s ten, and there’s twenty, and thirty, and more.
We’re with you in Rockland, we’re the wolf at the door.

But there’s no use in knockin’, we’re on the watch tower, watchin’.

You’re a fool, so am I! Ain’t it great? Never die!
We’re with you in Rockland, here’s a cot, have a lie.

Comments off

Merry Christmas, every one!

[ music | Michael Buble – Let It Snow ]

Merry Christmas, folks. Another year is nearly gone, and 2010 is right around the corner.  This year started like crap, got worse, and the past 4 months has shown me instead that life is still awesome, and I’m happier now than I may ever have been. I hope if nothing else, the new year is something to make all your lives brighter. Merry Christmas to everyone.

Waiting for Santa

Comments (1)

I’ve been a bad blogger…

[ music | Rilo Kiley – So Long ]

I’ve been neglecting my blog all year. For this,my dear readers, I apologize. I shall endeavour to be a better blogger in the new year. Between a really busy real life, and Twitter, I’ve just neglected to sit down and write up much. Even my previous post is a half-assed attempt, and I dislike doing that. For my day to day crap, check out my twitter feed.

However, to help make up for it, I’ll soon be making a kick ass end of year wrap-up post covering the best and worst everything from my viewpoint. Unlike some year-end wrap-ups, this one will be at the end of the year, between Christmas and New Years. I like to wait till the year is over to recap the year. 😉

Comments (4)