slash blog

A collection of thoughts and hacks

No More Snowflakes

Administrating a single server that’s your sole responsibility isn’t that much of a hassle but anyone who’s shared this responsibility with others or inherited machines manually configured by others knows without documentation will quickly tell you it’s a pain to work backwards from the finished server and maintain it going forward.

Getting bitten by this once or twice is OK, but as I get more involved in certain projects it’s becoming a stronger anti-pattern, and so I’m making a pledge to stop it.

Starting today new servers I admin will use some sort of automation, likely to be ansible. I’ve already started towards this with some Noisebridge projects and had some success. The ansible learning curve wasn’t that severe and I feel that at this point it’s mostly behind me to the point that I can be productive in it. As I write this blog post I’m provisioning a replacement personal server to host various services as well as my blog.

Using automation leaves behind a written, versioned record of the actions performed on the server: the software installed, the configuration files, users and other changes made. It’s code as documentation, meaning that you can simply share the playbook with anyone asking questions about the configuration.

It also allows you to easily “clone” existing infrastructure by creating a new VM and running the playbook such that the end state is the same of your production system. No gotchas, no “oh I made that change manually years ago and completely forgot to tell you or write it down” mistakes.

It’s 2016 and the tooling for all of this has substantially improved over the last few years. No more hacky bash scripts that fail in weird and wonderful ways, just stable automation software. If ever there was a time to make this change it is now.

NB There exists a noflake manifesto which Ross Duggan introduced in an earlier blog post of his in 2012. This post takes inspiration from both.

Running, Reading, Blue Hair, and Orbital Mechanics

I’ve lapsed yet again in the weekly writing requirement of the Iron Blogger project at Noisebridge and so feel compelled to give a core-dump type post to recount the happenings of the last two weeks.

I made a real breakthrough in my running in the last two weeks, after picking out some advice from /r/running on Reddit. I was absentmindedly browsing looking for training plans when I came upon a “beginners guide” of sorts which I scrolled though. All of it I was familiar with, except a very good point about pace: that you should run at such a pace as allows you to have a conversation throughout. Previously I’d run run intervals and push myself to the point of being out of breath, which sucked as I knew that I knew my legs could carry me further but my cardiovascular system could not. Slowing to a more comfortable pace has made the experience much more enjoyable: I find it easier to get into a groove where I can run for a while and let my mind wander. I’ve set a personal goal of running a 10k this summer with a friend, but have yet to nail down a particular race event. In the meantime I’m following a 10k training program and running 2-3 times a week.

I’ve started reading again after a lapse of many months. I’m reading two books simultaneously, the first being Tales of the City by Armisted Maupin, the second being Neal Stephenson’s latest work Seveneves. About the only thing the two share is the fact that they’re both fiction. Tales of the City is a great look into the San Francisco of yesteryear and the characters that enriched the culture that draws so many different groups to the city. I’m lucky enough to have friends in the city who have been here longer than the dominant technology and so get to experience some of the “counter culture” aspects to the city that I think many miss out on. Reading Tales of the City makes me want to contribute more to the “weird” side of the city as opposed to the “disruption” that everyone else seems intent on.

Seveneves is great near-future scifi where Stephenson blends the world we know and his magnificent fiction into an almost believable look at our future. The plot is centered around the human race’s attempts at survival in space after Earth is made uninhabitable by the breakup of the Moon and subsequent bombardment by meteorites. The combined nations of Earth have 2 years to create a viable space colony to carry on the human race, no easy task even without the social problems of having over a thousand humans in confined quarters in space. I’ve found this particularly hard to put down, keeping my Kindle with me to snatch a few pages here and there.

Reading about space in the generous detail that accompanies Stephenson’s fiction has me playing Kerbal Space Program again and brushing up on such subjects as orbital mechanics. I sat down and played a few hours of video games one night this week for the first time in what must be months, at least judging by the amount of Windows updates awaited me. KSP is massively entertaining with just the right mix of education and comedy. I’d highly recommend picking up a copy if you’re in any way interested. I’ve described it to people as a sandbox cartoon space program with all the mechanics intact. If that sounds good to you then you have many many hours of fun awaiting you.

I managed last Sunday to check off something I’ve wanted to do for a long while: dye my hair blue. I mentioned this wish briefly to Rubin while out drinking and he convinced me to do it that Sunday at his place. Rubin and Tilde were both incredibly generous in helping me out, and I’m incredibly pleased with the results. I’ve now joined the collection of folks you see wandering the streets of San Francisco with brightly coloured hair.

Rust Barcode Scanner v0.1, Freecon, and Exercise

This week I finished the first version of the barcode scanning flow I’ve been working on for Noisebridge. It’s a very simple Rust CLI that listens for input from a specified input device, and outputs only valid ISBN13 codes that it receives. Today I’ll work on a small Go program to take the ISBN numbers, looks them up and records the books in a PostgreSQL database. The next step after that is a small web service which allows people to search the books and see what we have at Noisebridge.

Working on this Rust program has been a lot of fun, and my first experience writing anything non-trivial in a compiled language. While there’s a lot to like about Rust, I found the initial learning curve a bit steep mainly due to the fact that the APIs were quite unstable pre 1.0.0. This meant that searching for solutions to issues often turned up results that were quite out of date due to things like core library functionality being split out into external crates. Post 1.0.0 with better documentation I don’t anticipate much hassle. Definitely this was a worthwhile learning experience doing some systems program for the first time, and it’s made me eager to do more of it in the future.

In other news, a band of like-minded individuals including myself have started conspiring to bring a Free Software unconference event to San Francisco named Freecon. After listening to Bradley Kuhn and others speak so elegantly about the place of Free Software in the world at Linux Fest North West in April I’ve wanted to involve myself more in the Free Culture, through softare communities and others. Currently, bar the very infrequent software patch to projects my sole contributions to Free Culture are through my work at Noisebridge and I’m hoping to branch out with this conference. We’re in the very early planning stages at the moment, but I’m excited to think about what we’ll put together. We’re aiming to hold the event on a single day one of the later weekends in October. If you’re interested and want to follow along you can sign up to our announcements list on the Freecon site, or you can join our mailing list

In non-(software|nerd) news, I went for a run this morning for the first time in a while. Since starting to cycle more in the last few weeks I’ve wanted to roll it over into starting running again in a bid to generally become more active and healthy. I’m very fortunate in that I live only two blocks away from Golden Gate Park, which provides an amazing destination for runs of all lengths. Running through the park and enjoying the weather is a really simple pleasure which I hope to avail more of. I’m following the “Couch to 5k” program and recording the runs as I take them, so we’ll see how I progress over the coming weeks. Already I’ve noticed a general improvement in my fitness from the cycling I’ve been doing: getting quicker on the regular routes that I take while being less out of breath on the other side. All progress is good progress, and it’s exciting to be making some.

Breaking Radio Silence

It’s been a while since I’ve had the opportunity to sit down and write a nicely thought out blog post, but in the complete absence of any rhythm in recent postings I figured I’d write down some thoughts about the last 2 weeks.

I’ve been making slow (mostly due to time commitments), but good progress on the USB barcode scanner project I’ve been hacking on for Noisebridge. One of the frustrations in choosing Rust for this project was the lack of API stability in pre-1.0.0 Rust. This manifested itself in many ways, but mostly it was documentation being out of date and confusing given breaking API changes, or refactors where functionality that was previously in “core” being moved out into their own external crates.

That said, Rust recently went 1.0.0, and with the new stability and renewed energy going into documentation I’m even more excited to continue hacking on things using it. I’ve found each problem on the way with this project interesting enough to get lost in the rabbit hole of documentation to see what the Rust way is to do things, which has provide for a very enjoyable learning experience.

In Noisebridge news, the past two weekends have been very busy for the community, first with the Stupid Shit No-One Needs and Useless Technology Hackathon which was a great success, followed by Maker Faire this week. The hackathon was a 2 day event taking place at multiple venues in the Bay Area, at Noisebridge, Sudoroom and Mozilla. I arrived midway through Saturday to find the space packed with people having fun making things, showing them off to others and getting a few laughs. I lost count of how many people came through Noisebridge in the hours I was there. It struck me that it was a particularly diverse crowd for such an event, and I was delighted to see Noisebridge take part in it. I’m hopeful something like it will happen again in the near future.

I have a bunch of projects taking my time at the moment, some of which I’m going to be very excited to write about when they’re a bit more solidified and presentable, everything ranging from SDR hacks, to events and conferences, with Noisebridge infrastructure mixed in the lot. I’m anticipating that I’ll have no shortage of things to write about :)

Linux Fest North West & Hackers on a Train Recap

I’ve been neglecting my weekly blog post the last few weeks, but I’m back! The last few weeks and weekends have been busy with fun stuff.

The workshop I gave on Data Security for Journalists at the CPJ conference was really wonderful fun. It was great to meet Cyrus and Micah and hang out with them teaching journalists how to use security tools to protect both themselves and their sources. The workshop itself was a great success, with all attendees walking away with both a working knowledge and a suite of tools to help them in their daily work. Feedback after the workshop was really positive as well. All in all a success I’d very much like to recreate again.

The Sunday of that same weekend I attended my first B-Sides in SF. A few Noisebridge folks ended up attending together. The highlight of the day was all of us competing in the CTF that was running at the event. I’ve dabbled in CTF events before, but this was the first one I’ve really competed in before. The Noisebridge team ended up racking up a respectable point tally before the end of the day. It was fun tinkering around trying to break the web-app challenges.

That following Wednesday was the beginning of the Hackers on a Train journey to Linux Fest North West in Bellingham, WA. We left Oakland on Wednesday night on the Coast Starlight Amtrak route. I’d not ever tried train travel in the US before, but was very pleasantly surprised by the experience. It was a thoroughly enjoyable ~22 hour ride to Seattle. We spent the majority of our time in the observation car taking in the countryside and hacking a thing. A very relaxing way to travel. I’d like to travel by train more in the future when time allows.

Linux Fest North West was my first experience of a Free Software conference, and I was really blown away by the community there. The exhibition floor had stands from a bunch of Free Software communities, and it was great getting to chat with folks about their work in Free and Open Source Software. Many had been in the community for a long time, and it was inspiring to see the work they’ve contributed to making this software and community more awesome for folks.

A definite highlight of the weekend was attending a talk by Bradley Kuhn on the future of Copyleft. Bradley is a key figure in the copyleft community, working with the Free Software Conservancy and others to protect Free Software in the world. I was very much inspired by Bradley’s history with the community and his rousing words on the need to defend Free Software and its use in the world on a continual basis. I walked away from the talk with a much better understanding of the Free Software licenses, their history, importance and place in the ecosystem.

In project news, I’ve been dabbling in Rust and systems programming recently, writing a small daemon to handle input from a USB barcode scanner to be used in a book cataloguing setup at Noisebridge. It’s been a refreshing change of pace to write such code, diving into a new language like Rust, and learn a new thing at every step of the way. You can follow my (admittedly slow) progress on Github. Having spent the vast majority of my time writing code in interpreted, duck-typed languages having a compiler to contend with / help guide you is a fun change. I’m finding Rust a really fun language to learn, and in particular the #rust IRC channel to be an awesome community learning resource. All in all it’s making me want to write more learn more about this style of programming.

Given things are a little less busy I’m hoping to get back into the weekly blog schedule. I’ve missed taking the time to write something each week, and don’t want to let the habit die. Watch this space for more.

Thoughts on Teaching Crypto to Journalists

Last Saturday I gave a workshop on Data Security to a small group of Bay Area journalists at Noisebridge. The workshop was a small-group test of the same material that’ll be used at a larger workshop I’m helping to lead at the Committe to Project Journalism’s SF conference later this month.

Overall it was a great success and a really awesome experience, with 4 journalists walking away with a fully functional email encryption setup, as well as a good working knowledge on the concepts underpinning it.

I wanted to write this to share some lessons that I learned through this first attempt at teaching. It was a fun experience, and something I’d like to improve upon and repeat in the future.

Use scenarios

One of the biggest lessons I came away from the workshop with was the value of using scenarios in teaching. Traditional cryptography teaching material almost always uses Alice and Bob to describe the people involved, but when then number of participants grows it can be hard for those new to the ideas to follow who’s who, and what their role in the scenario is.

Describing a short scenario such as “Your source at $MEGACORP has found that the latest SOMA high-rise complex isn’t using the required rebar in construction and wants to blow the whistle” immediately provides you with a set of characters that are easy to relate to, but still describe the technical concept at hand. While it took a few questions back and forth with Alice and Bob to clarify their respective roles, things clicked almost at once once the characters changed.

Talk slow, teach slower.

As is often the case when explaining technical concepts to a non-technical audience it can be very hard to put oneself in the audience’s shoes, especially when it comes to presumed knowledge. The absolute best step here is to just presume nothing, and go from the very basics.

An example of this was that I started explaining where email cryptography layers on top of email by jumping straight into explaining enigmail, without first checking that everyone was comfortable with the idea of using another email client like Thunderbird. An honest mistake, and an easy one to clear up, but once this was covered everyone had a much better understanding of how email crypto is an optional layer on top of their daily work.

Teach in lockstep

As part of the workshop I wanted everyone to come away with a fully functional email cryptography setup on their laptop. The first major part of this was installing all the requisite tools on everyone’s machines.

Rather than just give everyone instructions and have folks run through them independently, going through each step together in a lockstop fashion with everyone helping each other proved to be really beneficial, especially with folks offering explanations of the things they were doing, further reinforcing their understanding.

You’ll never cover as much material as you’d like

I originally came into the workshop hoping to cover all of the following in a short 3 hours.

  • PGP
  • TextSecure / Signal
  • OTR
  • Tails

In the end however we covered only the first of these. The reason for this was a combination of all of the above lessons, but I think it’s important to state on its own. In my opinion it was much more beneficial to work slower through PGP, and getting everyone set up than it would have been to give only brief overviews on each topic with the expectation that folks would then pursue them after the workshop on their own.

The reality of the situation is that even with such help, these tools are still incredibly daunting for non-technical people to approach on their own for the first time, especially journalists who constantly have to defend their time from every email and call they get from the source with “mind-blowing material that’ll change the world”. One need look no further than Edward Snowden’s attempts to get Glenn Greenwald to setup a PGP key, eventually having to contact Laura Poitras separately to get Glenn involved.

Until the tools become usable such that it doesn’t require a workshop to get started using them, I think the most benefit that people will get from such events is from more thoroughly covering a smaller selection of topics with hands-on help than by giving a lecture overview of acronyms and topics that’ll all too soon be forgotten.

In summary: you should try it

I had a lot of fun giving this workshop, and it definitely made me very excited about the upcoming larger event at the CPJ conference in SF. Outside of this I think this smaller workshop pattern is definitely one that could be repeated, and that’s exactly what I plan to do.

Given the ease with which digital communications can be subpoenaed, intercepted, and meddled with journalists and their sources face innumerable difficulties in communicating with security and / or anonymity. While they’d like to, most don’t have the technical expertise to start. Sharing knowledge like this is an immense help help to journalists looking to shine light on important issues, and also as importantly protect their sources.

Cycling Bikes Is Fun

I started riding my bike to work the other day, and it’s very enjoyable. I’ve been feeling a bit sedentary in recent times, so a more active morning and evening commute is a really welcome change. I’d kinda forgotten that nice feeling you get post-exercise, and it’s made me realise that I should do this more often.

I find it easy to fill time with non-exercise related things, so coupling exercise with my daily commute will hopefully facilitate this becoming a habit. Usually with new changes like this I’m not very successful in consistently carving out the time to dedicate to it. Swapping out a MUNI ride for cycling is a perfect alternative.

The route to work is a real downhill joy in the morning, and it most definitely helps with waking up mentally. By the time I get to FIDI I feel ramped up for the day.

The homeward leg is relatively painless, the only difficult point being the last half mile from the panhandle up to my apartment on a hill. In hindsight a single gear might not always be the best option, but it’s not all that much of a big deal.

I’m going to give this a few weeks and see how it goes. I’m cautiously optimistic that I can make this a productive, healthy habit.

Anti-pattern Checkin Week #3

It’s been 3 weeks since Noisebridge’s Iron Blogger kicked off. While I’m happy to have kept myself to a schedule of writing at least once a week, I’m still trying to beat the anti-pattern of scrambling to write something last thing on a Sunday evening.

Thankfully, I have a few things I’m working on that I’d like to write about in more detail. I’m hoping to flesh out a few post stubs to keep on hand during the coming weeks such that I’ll never be short of ideas about which to write.

Drobo recovery: in progress

A quick update on the Drobo recovery adventure of 2015. I’ve been slowly moving data off the Drobo onto an external HDD that I’m using as a temporary store, and as of yesterday have a 90% assembled FreeNAS box ready to replace it.

I went slightly over budget on the FreeNAS replacement, ending up with a very compact, quiet i3 4130T build which I think shall last a considerable amount of time. I got a good deal on a Fractal Node 304 case which required a change in the partlist to be compatible with the smaller Mini-ITX formfactor. The resulting machine without the 5 HDDs is incredibly quiet and small. It’s a marvel to see everything fit in there even without the drives.

I’m hoping by the end of this week to have the data completely copied off the Drobo and the drives installed in the NAS rig, but progress here isn’t guaranteed with hours coming in sporadic chunks. Either way I’m excited to have the finished project running at home.

Data Security for Journalists workshop

I’ve begun putting together a small amount of material to accompany the Data Security for Journalists workshop that I’ll be helping to run in April. I’ll be hosting a much smaller dry-run at Noisebridge this coming Saturday with 6 journalist friend-of-friends to better judge the topics and ideas I have about what to do with the time allotted.

I’m glad of the opportunity to write some technical documentation, a skill which I think is very valuable and one which I’d like to develop further. Writing an introduction to public key cryptography and then giving it to non-technical friends for feedback has been a really fun learning experience. One really must be very comfortable with such topics to be able to effectively communicate them with a non-technical audience.

Building things.

I got a small amount of time this week to work on K-9, the art car that QCCB built for Burning Man 2014. K-9 had an outing at a Doctor Who themed art show in SF which required some amount of preparation beforehand. It was a lot of fun to do some hands-on work and escape computers for a few hours. It reminded me how much fun it was to be a part of creating K-9 in the first place, and that I should try to build more physical things alongside my computer-y endeavours.

Don't Lock Your Data in Proprietary Products: or, Why I'm Replacing My Drobo With a FreeNAS Box

PSA: Don’t trust your valued data to proprietary products. When they break you’ll be up a proverbial creek without a paddle.

The Drobo FS honeymoon period

Back in 2011 I bought a Drobo FS with the intention of using it as a home backup solution. The features it offered were exactly what I was looking for: some RAID-like setup promising single parity as well as a neat ability to setup network Time Machine shares. I filled it with 5 2TB drives and let it run.

All worked well for the first 18 months. The only complaint I had was that the CPU was a bit underpowered, making certain DroboApps (packaged Linux services you can run on the Drobo like dropbear, upnp-servers etc…) a bit slow. Still, it was a neat little package that was performing its main duty well: holding my backups, media, photos, recordings and other important things that you don’t want to lose.

Oops, I lost the data

Then one day, everything stopped working. To the best of my recollection the issue was caused by a power outage at home which resulted in a corrupted disk pack. The drobo would no longer boot properly, leaving my data inaccessible on the drives. The proprietary RAID-like setup meant that even with another machine, I’d not be able to retrieve my precious data.

I went back and forth with Drobo support for a long time, and while they were very willing and accommodating in helping me, the data remained lost, out of reach. After many frustrating attempts at reviving it, in the hopes that some future firmware update would liberate my data, I decided to just cut my loss and pack it into a closet where it has sat for almost 18 months.

Next: a FOSS solution

I decided this weekend to build a new FreeNAS server at home to replace the Drobo. Through some miracle I’ve managed to kick the Drobo into successfully healing the corrupted pack. It was, and I hate to admit this, by complete accident. While trying to boot the Drobo into read-only mode once again (the same approach that’d failed 18 months earlier), it recognized the failure and began fixing it. ~24 hours later I have access to my data again, but using a solution that I don’t trust as far as I could throw it. I’m going to document the build and setup here such that others might find value in it.

The parts are ordered, and hopefully by this coming weekend I’ll have some progress to share.

Iron Blogger: It Begins

I finally, this past weekend, got around to implementing the Iron Blogger group that I’d been hoping to start at Noisebridge before Christmas (better late than never). This post is actually the first of my weekly obligations. Watch this space for more blogging.

Things are going well. Life is busy on many fronts. The plans that I had for Linux Fest North West and, more specifically Hackers on a Train are coming along nicely. Torrie, Jay and I have our travel booked, and others have expressed interested and marked the dates in their calendar.

Noisebridge has had 2 really wonderful fundraising events in the last few weeks, first the revival of 5 minutes of fame, and the second an awesome party which have brought some old hands and many new faces to the space. It’s encouraging to see the reactions of people walking in the door. Rubin even proclaimed it a “hackerspace” once again. It feels like we’re on the right track. We still have many events and fundraising to do to keep the doors open, but these first two have made me very optimistic.

I’ve become involved in putting on a workshop teaching the use of basic cryptographic tools to journalists at a conference run by the Committee to Protect Journalists in April. It’s the first time I’ve been involved in something like this, so while I’m excited for the opportunity I’m slightly nervous about my lack of experience. That said there’s a first time for everything, and I’m confident it’ll be useful for those who attend. There’s plenty of helpful material online from groups like the Cryptoparty and the EFF’s Surveillance Self Defense project which makes planning easier. I’ve long held strong views on these issues and I’m glad for the opportunity to put them into action.

I hacked a bit last week on Noisebridge’s RFID access control system, a large part of which is written in Go. I’d heard much about Go, but hadn’t yet had a chance to do anything really useful with it. It was fun, and is now firmly on my radar as a technology to use in upcoming personal projects.