• Archives

  • Categories:

  • Other profiles

Wedding and week in Florida

All this travel is leaving me in the unfortunate position of having a growing pile of blog posts queuing up, which will only get worse as the OpenStack Summit continues this week, so I better get these out! I’m now in Paris for the summit, but last week I was in Florida for MJ’s cousin Stephanie’s wedding.

I arrived on Friday afternoon from Raleigh and MJ picked me up at the airport, getting us to the hotel just in time to get changed for a family and friends gathering the evening before the wedding.

Saturday we were able to enjoy the beach and pools at the hotel with some of MJ’s cousins. The weather was great, even the humidity was quite low, relative to what I tend to expect from Florida.

As the day wound down, we got ready for the wedding!

The ceremony and reception took place at a beautiful country club not far from the hotel. As an attendee, it seemed like everything went very well. The reception was fun, lots of great food, a fun, sparkly signature drink and some stunning centerpieces decorating the dinner tables. I even danced a little.

Unfortunately I picked up a cold somewhere along the way, and spent all of Sunday in bed while MJ spent more time with family and pools. By Monday I was feeling a bit better and was able to see MJ off and get moved over to the beach motel where I spent the rest of the week.

My beach motel wasn’t the greatest place, but it was inexpensive, clean and ultimately quite tolerable. The plan to stay in Florida, in spite of my general “I don’t like Florida” attitude, was to avoid going all the way back to California prior to my Paris trip. And I have to say, with nice October weather and the views at sunset, I think it was the right choice.

My days were spent catching up with work post-conference and prepare for the summit this week. Thankfully it wasn’t very hot out, so I was able to open the windows during the day and let fresh air into my rooms. I also made plans throughout the week to visit with family in the area, managing to meet up with my cousin Shannon and her family, my Aunt Pam, and my Aunt Meg and cousin Melissa throughout the week.


At dinner with Shannon, Rich & Frankie

I also was able to take some long lunch breaks to enjoy a few quick dips in the ocean.

The San Francisco Giants won the World Series while I was in Florida too! I was able to watch the games in my room each night. I was disappointed not to be in town for the win, as the whole city explodes in celebration when there’s a win like this. My week wrapped up on Friday when I checked out of the motel and headed toward the airport for my redeye flight to Paris. And since I was also disappointed to be missing Halloween in San Francisco again, I dressed up for my flight, as Carmen Sandiego.

All Things Open 2014

From Oct 22-23rd I had the pleasure of speaking at and attending All Things Open in Raleigh, North Carolina. Of all the conferences I’ve attended this year, this conference is one of the most amazing when it comes to how well they treated their speakers. When I submitted my talk I received an email from the conference organizer thanking me for the submission. Frequent emails were sent keeping us informed about all the speaker-focused conference details. Leading up to the event I woke up one morning to this flattering profile on their news feed. A series of interviews was also published by the OpenSource.com folks featuring speakers. Once there, I was thanked about 100 times throughout the 2 day event. In short, they really did a remarkable job making me feel valued.

Thankfulness aside, the conference was top notch. Several months back I read The foundation for an open source city by Jason Hibbets so I was excited to go to Raleigh (where much of the work Hibbets talked about centered around) and doubly amused when Jason said hello to me and I got to say “hey, I read your book!” During the conference introduction they said the attendence last year (their first year) was around 700 and that they were looking at 1,100 this year. The conference was opened by Raleigh Mayor Nancy McFarlane, which was pretty exciting, I’d seen cities send CTOs or supervisors, but the having the mayor herself show up was quite the showing of support.

After her keynote came Jeffrey Hammond, VP & Principal Analyst at Forrester Research. I really enjoyed the statistics his company put together regarding the amount of open source software being used today. For instance, of developers surveyed they learned that 4/5 of them are using open source software and 73% of them are programming outside of their paid job, 27% on open source.

Right after the keynotes I headed downstairs to give my talk, Open Source Systems Administration. A blending of my passion for open source and love of systems administration, this is one of my favorite talks to give, I really enjoy being able to present on how the OpenStack infrastructure itself is an open source project. It was a lot of fun chatting with people throughout the rest of the conference who had attended (or missed) my talk, while there is less surprise these days that a project would open source an infrastructure, there’s a lot of interest in learning that there are project which actually have and how we’ve done it. Slides from my talk here: ATO-opensource_sysadmin.pdf (2.3M).


Giving my talk! Thanks to Charlie Reisinger for this photo.

The schedule made it hard to select talks, but I next decided to head over to the Design track to learn from Garth Braithwaite why Open Source Needs Design. I’ll start off by saying that it’s wonderful that there are some designers participating in open source these days, but as Garth points out in his talk they are generally: paid by a company as a designer to focus on the product (open sourceyness of it doesn’t matter, it’s a job), a designer friend of someone in the project who is helping out or a developer on the project who happens to have some design expertise (or is willing to get some in order to help the project). He explored some of the history of how developers made their way to open source and the tools we used, and explained that the “story” doesn’t exist for designers, why would they get involved? They’re not fixing a printer or solving some tricky problem. The tools for open collaboration for designers also don’t really exist, popular sites for design sharing like Dribbble don’t have source upload options and portfolio sites like BeHance lack any ability for collaboration. The new DesignOpen.org seeks to help change that, so it was interesting to learn about that. From there he detailed different types of design work, UX, IxD and UI and the tools and deliverables for each type of work. As someone who really has never worked with design it was an interesting tour of that space. His slides from the talk are available here: speakerdeck.com/garthdb/open-source-needs-design (first few slides are a image-full, but stick with it, some great slides with bullet points come later!).

Then it was off to see Lessons Learned with Distributed Systems at bit.ly presented by Sean O’Connor (it was a pleasure to meet him and colleague Peter Herndon during the keynote earlier in the day). The talk centered around some of the concerns when architecting systems at scale, from time syncronization to having codebases that are debuggable. At bit.ly they adopted a codebase that is broken out into many small pieces, allowing ops to dig into and learn about specific components when something goes wrong, not necessarily having to learn everything all at once in order to do their job effectively. He also went into how they’ve broken their workload up into what has to be done synchronously and what can be shifted into an asynchronous job, which is preferred because it’s easier to do well. Finally, he talked some about how they deal with failure, starting off with actually having a plan for failure, and doing things like back offs, where the retries end up spaced out over time rather than hammering the service constantly until it has returned.

After lunch I decided to check out the Messaging Standards and Systems – AMQP & RabbitMQ talk by Gavin M. Roy. I’ve used RabbitMQ a fair amount, but that doesn’t mean I’ve ever paid attention to AMQP (Advanced Message Queuing Protocol), I was pretty surprised to learn that releases 0-8 and 0-9-1 are very different the 1.0 release and are effectively overseen by different people, with many users still intentionally on 0-9-1. Good to know, I imagine that causes a ridiculous amount of confusion. He went through some of the architecture of how RabbitMQ can be used and things it does to “fix” issues encountered with the default AMQP 0-9-1. Slides from his talk here speakerdeck.com/gmr/messaging-standards-and-systems-amqp-and-rabbitmq (the exchange slides about halfway through are quite helpful).

I was then off to Saving the World with Open Source and Science presented by Dr. Marcus Hanwell. Given my job working on OpenStack, I perhaps have the distinct benefit of being exposed to scientists who understand how to store, process and present big data, plus who understand open source. I assumed this ubiquitous, so this talk was quite the wake up call. Not only are publicly-funded papers not available for free (perhaps a whole different rant), the papers often don’t have enough data for the results to be reproducible. Sources from which data was processed aren’t released (whether it be raw data, source code used to make computations or, seriously, an Excel spreadsheet with some data+formulas), images are shrunk and stripped of all metadata so it can be impossible to determine whether you’re actually seeing the same thing. Worse, most institutions have no way to index this source material at all, so something as simple as a harddrive failure on a laptop can mean loss of this precious data. Wow, how depressing. But the talk was actually a call for action in this space. As technologists there are things we can do to provide solutions to scientists, and scientists working in research can make social changes so that releasing full sources, code and more becomes something valued and validation of results is something that once again becomes central to all scientific research.

Day one completed with a keynote by Doug Cutting he titled “Pax Data” which was a fascinating look into the world we’re building where the collection of data is What We Do. He began by talking about how in most science fiction the collectors of data end up being the Bad Guys in a future dystopia, but the fact is that sectors from Education to Healthcare to Climate can benefit from the collection and analysis of big data. He posted the question to the audience: How do we do this without becoming those Bad Guys? He admitted not having a full answer, but provided some guidance on key things that would be required, including transparency, best practices around data handling, definition of data usage abuse so we can protect against it, and either government or industry oversight and/or regulation. Fascinating talk for me, particularly as I was in the middle of reading both a SciFi dystopia book where big data becomes really scary (The Circle by David Eggers) and non-fiction book about our overuse of technology (Program or be Programmed).

Day 2! Keynotes began with a talk by James Pearce of Facebook. I know Facebook is pretty much built on open source (just like everyone else) but this talk was about the open source program he and his team have built within Facebook starting about a year ago. As is standard for many companies starting with open source, they’d just “throw things over the wall” and expect the code to be useful to the community. It wasn’t. So they then began seriously working to develop the code the were open sourcing, assigning people internally to be the caretakers of projects, judging the health of projects based on metrics like forks and commits from community members outside of Facebook. They also run much of the same code versions internally as they release in the community. Github profile for Facebook is here: https://github.com/facebook. Very nice work!

The next keynote was by DeLisa Alexander of Red Hat on Women in Open Source. She started out with a history lesson about how the first real programmers were women and stressed why diversity is important in our industry. Stories about how the most successful women in open source have had encouragement of some form from their peers, and how important it is that everyone in the audience seek to do that with newcomers to their community, particularly women. It was also interesting to hear her talk about how children now often think of computers as opaque black boxes that they can’t influence, so it’s important to engage children (including girls) at a young age to teach them that they can make changes to the software and platforms they use.

Alexander also hosted a panel at lunch which I participated in on this topic. I was really honored to be a part of the panel, it was packed with very successful women in tech and open source. Jen Wike Huger wrote up some of her notes in a great article here: Keys to diversity in tech are more simple than you think. My own biggest takeaway from the panel was the realization that everyone on the panel has spent a significant amount of time being a mentor in some formal capacity. We’ve all supported students and other women in technology via organizations that we either work or volunteer for, or run ourselves.

Getting back to sessions, I went to Steven Vaughan-Nichols’ talk on Open Source, Marketing, and Using the Press. Now, technically I’m the Marketing Lead for Xubuntu, but I somewhat joke to people that it’s “only because I know how to use Twitter.” Amusingly, during his talk he covered people just like myself, project contributors who end up with the Marketing role. I gained a number of great insights from this talk, including defining your marketing audience properly – there’s your community and then there’s the rest of the world. Tips to knowing your customer, maybe we should do a more formal survey in Xubuntu about some of the decisions we make rather than relying upon sporadic social media feedback and expecting users to participate in development discussions? He also drove home the importance of branding, which thanks to our logo designer Pasi Lallinaho I believe we have done a good job of. There was also a crash course in communicating with the press: know who you’re contacting and what their focus is, be clear and concise in emails and explain the context in which your news is exciting. Oh, and be friendly and reply promptly when reporters contact you. I also realized I should add our press contact to our website, that’s a good idea! I have some updates to make to the Xubuntu Marketing blueprint this cycle.

Perhaps one of my favorite talks of the even was presented by Dr. Megan Squire: Case Study: We’re Watching You: How and Why Researchers Study Open Source And What We’ve Found So Far. I think what I found most interesting is that while I see poll from time to time put out by people claiming to do research on open source, I never see the results of that research. Using what I now know from Dr. Marcus Hanwell (many academic papers are locked behind journal pay walls) this suddenly makes sense. But Dr. Squire’s talk dove into the other side of research that doesn’t include polls: research done on data, or “artifacts” that open source projects create. Artifacts are pretty much anything that is public as a result of a project existing, from obvious things like code to the public communication methods we use like IRC and mailing lists. This is what is at the heart of a duo of websites she runs, the first being FLOSSmole which connects well-formatted data about projects with researches interested in doing datamining against it, and FLOSShub which is a collection of papers she’s collected about open source so it’s all in one place and we can see what kind of research is being done. Aside from her great presentation style, I think what made this one of my favorites was the fact that I didn’t know this was happening. I make FOSS artifacts all day long, both in my day job and with my open source hobbies, and sure I know it’s out there for anyone to find, piles of IRC logs, code reviews, emails, but learning that academics are actively processing them is another thing entirely. For instance, to take an example from a project I work on, I had no idea this existed: Estimating Development Effort in Free/Open Source Software Projects by Mining Software Repositories: A Case Study of OpenStack. It made me a bit tin-foil-hat for about 5 minutes until I once again realized that I’m not just fine, but happy to be putting my work out there. Huge thanks to her for doing this presentation and maintaining these really valuable websites.

Slides from her presentation are up on Google docs here and are well worth the browse for examples she uses to illustrate how our artifacts are being used in research.

After lunch I attended my last three talks for the conference, the first one being Software Development as a Civic Service presented by Ben Balter. I’ve attended a number of civic hacking focused talks at events over the past couple years, but this one wasn’t strictly talking about a specific project or organization in this space. Instead he focused on the challenges that confront governments and us as technologists as we attempt to enter the government space, and led to one of my favorite (sad!) slides of the event, in which you will note that doing anything remotely modern (use of public package repositories, configuration management or source control) doesn’t factor in:

He talked about how some government organizations are simply blinded by proprietary sales talk and FUD around open source, while others actually are bound by specific governmental requirements in the software that industries have figured out, but open source projects don’t think to include (ie – an Open Source CMS may get us 99% of us there, but this company is offering something that satisfies everything because it’s their job to do so). He also talked some about the “Command and Control” structure inside of government and how transparency can often be seen as a liability rather than the strength that we’ve come to trust in within the open source community. He wrapped up with some success stories from the government, like petitions.whitehouse.gov and GOV.UK and shared some stats about the increase of known government employees collaborating on Github.

The next talk was by Phil Shapiro on Open Sourcing the Public Library. To begin his talk he talked about how open source has a major opportunity as libraries move from the analog to digital space. He then moved into a fact he wanted to stress: libraries are owned by all of us. There is an effort to transform them from the community “reading room” into the community “living room” where people share ideas and collaborate on projects, bringing in more educational resources in the form of classes and the building of maker spaces. I love this idea, I find Hackerspaces to be unintentionally hostile places for many young women, so providing a different option to accomplish similar goals is appealing to me. I think what struck me most about this was how “open sourcey” it felt, people coming together to build something new together in the open in their community, it’s why I work on any of this at all. He shared a link of some collected writings about the future of Libraries here: https://sites.google.com/site/librarywritings/

The final talk of the day I attended was Your Company Culture is Awesome (But is Company Culture a Lie?) by Pamela Vickers. In her talk she identified the trend in technology of offering “perks” in lieu of an actual healthy work environment for workers. These perks often end up masking real underlying unhappiness for employees, and ultimately lead to loss of talent. She suggested that companies take a step back from their pile of perks and look to make sure they’re actually meeting the core needs of their employees. Are your developers happy? How do you know? Are you asking them? You should, and your employees should trust you to be honest with you and to at least professionally acknowledge their feedback. She also highlighted some of the key places where companies fall down on making their developers happy, including forcing them to use the wrong tools, upsetting a healthy work-life balance, giving them too much work or projects that don’t feel achievable and giving them boring or unimportant projects.

To wrap this up, huge thanks to everyone who worked on and participated in this conference. As a conference sponsor, my employer (HP) had a booth, but unfortunately I was the only one who was able to attend. I spent breaks and lunches at the booth (leaving a friendly note when I was away) and had some great chats with folks looking for Python jobs and who were more generally interested in the work we’re doing in the open source space. It still can strike people as unusual that HP is so committed to open source, so it’s nice to be available to not only give numbers, but be a living, breathing example of someone HP pays to contribute to open source.

3 weeks at home

I am sitting in a hotel room in Raleigh where I’m staying for a conference, but prior to this I had a full 3 weeks at home! I was the longest stretch I’ve had in months, even my gallbladder removal surgery didn’t afford me a full 3 weeks. Unfortunately during this blessed 3 weeks home MJ was out of town for a full 2 weeks of it. It also decided to be summer time in San Francisco (typical of early October) with temperatures rising to 90F for several days and our condo not cooling off. Some days it made work a challenge as I sometimes fled to coffee shops. The cats didn’t seem amused by this either.

The time at home alone did give me a chance to chill out at home and listen to the Giants playoff games on the little AM radio I had set up in our living room. As any good pseudo-fan does I only loosely keep up with the team during the actual season, going to actual games only here and there as I have the opportunity, which I didn’t this year (too much travel + gallbladder). It felt nice to sit and listen to the games as I got some work done in the evenings. I did learn how much modern technology gets in the way of AM reception though, as I listened to the quality tank when I turned on the track lighting in my living room or random times when my highrise neighbors must have been doing something.

Fleet week also came to San Francisco while I was home. I think I’ve only actually been in town for it twice, so it was a nice treat. To add to the fun I was meeting up with a friend to work on some OpenStack stuff on Sunday when they were doing their final show and her office offers amazing floor to ceiling windows with a stunning view of the bay. Perfect for watching the show!

I also did manage to get out for some non-work social time with a couple friends, and finally made it out to Off the Grid in the Marina for some street food adventuring. I hadn’t been before because I’m not the biggest fan of food trucks, the food is fine but you end up standing while eating, making a mess, and not getting a meal for all that cheaper than you would if you just went to a proper restaurant with tables. Maybe I’m just a giant snob, but it was an interesting experience, and I got to take the cable car home, so that’s always fun.

And now Raleigh. I’m here for All Things Open which I’ll be blogging about soon. This kicked off about 3 weeks away from home, so I had to pack accordingly:

After Raleigh I’ll be flying to Miami for a cousin’s wedding, then staying several extra days in a beach hotel where I’ll be working (and taking breaks to visit the ocean!). At the end of the week I’m flying to Paris for the OpenStack Summit for a week. I’ve never been to Paris before so I’m really looking forward to that. When the conference wraps up I’m flying back stateside for another wedding for a family member, this time in Philadelphia. So during this time I’ll get to see MJ twice, as we meet in cities for weddings. Thankfully I head home after that, but then we’re off for a proper vacation a few days later – to Jamaica! Then maybe I’ll spend all of December in a stay-at-home coma, but I’ll probably end up going somewhere because apparently I really like airplanes. Plus December would be the only month I didn’t fly, and I can’t have that.

33rd Birthday Weekend

I’m a big fan of trying new things and places, so it came as a surprise that when I decided upon a birthday getaway this past weekend we decided to go back to the Resort at Squaw Creek, where we had been last year. It wasn’t just travel exhaustion that made us choose this one, we knew we wanted to get some work done during the weekend and the suite-style was great for that. Honestly we love everything about this place – beautiful views, amazing pools, good food. The price was also right for a quick getaway.

The drive up was a long one, Friday evening traffic combined with a thunderstorm. We stopped for dinner at Cottonwood Restaurant in Truckee. By the time we arrived at the driveway to the resort the rain had transformed… what is that, slush? By the time we got to the front door it was properly snowing!

Saturday morning we had breakfast brought to our room (heaven!) and enjoyed the stunning view outside our window.

The rain kept us inside for most of the day, which was wonderful. I was able to get some work done on my book (as planned!) and MJ did a bunch of research into our first proper vacation of the year coming up in November. Fireplace, hot chocolate, the man I love, perfect!

As 4PM rolled around the rain tapered off and we went down to the pool. It was 45F degrees out, so not exactly swimming weather, but the pools were heated and the trio of hot tubs were a popular spot for other folks visiting for the weekend. It turned out wonderful, particularly with the standard warm fall we’re having in San Francisco. That evening we had wonderful dinner (and dessert!) at the on site restaurant.

Sunday was even more rainy. We took advantage of their option to pay an extra $85 to get an 8pm checkout, giving us the whole day to enjoy before we had to go home. The rain did end up keeping us from the pool, but I did take a 2 mile walk through the woods with an umbrella after lunch. In spite of the rain, it was a beautiful walk up the sometimes steep and rocky terrain through the woods.

Alas, it had to end. On our way out we stopped at FiftyFifty Brewing Company for a casual dinner. They had the most amazing mussels appetizer, I kind of want to go back to have that again. Dinner wrapped up with some cake!

Fortunately the drive home was quicker (and drier!) than our drive to the mountains had been and we got in shortly before 1AM.

My actual 33rd birthday was on Monday. I ended up making plans with a friend who was in town to celebrate her own birthday the following day. We met up at the San Francisco Zoo around 11AM and I finally got to meet the wolverines! Even better, we caught them as a keeper was putting out some treats, so we got to see them uncharacteristically bounding about their enclosure as they attacked the treat bags that were put out for them. Alas, in spite of staying until the opening of the Lion House, I still managed to miss the sneaky two-toed sloth who decided to hide from me.

We wrapped up the afternoon with lunch over at the Beach Chalet.

It was a great birthday weekend+birthday, aside from the whole turning 33 part. Getting older hasn’t tended to bother me, but time is passing too quickly, much still to do.

PuppetConf 2014

Wow, so many conferences lately! Fortunately for me, PuppetConf was local so I didn’t need to catch any flights or deal with hotel hassle, it was just a 2 block walk from home each day.

My focus for this conference was learning more about how people are using code-driven infrastructures similar to ours in the OpenStack Infrastructure project and meeting up with some colleagues, several of whom I’ve only communicated with online. I succeeded on both counts and it ended up being a great conference for me.

There was a keynote by Gene Kim, he is one of the authors of the “devops novel” The Phoenix Project, which I first learned about from my colleague Robert Collins. His talk focused around the book, as The Phoenix Project: Lessons Learned. In spite of having read the book, it was great to hear from Kim on the topic more directly as he talked about technical debt and outlined his 4 top lessons learned:

  • The business value of DevOps is higher than we thought.
  • DevOps Is As Good For Dev… …As It Is For Ops
  • The Need For High-Trust Management (can’t bog people down)
  • DevOps is not just for the unicorns… DevOps is for horses, too. (ie – not just for tech stars like Facebook)

Talk slides here.

The next keynote was by Kate Matsudaira of Popforms who gave a talk titled Trust Me. I wasn’t sure what to expect with this one, but I was pleasantly surprised. She covered some of what one may call “soft skills” in the tech industry, including helping others, supporting your colleagues and in general being a resourceful person who people enjoy working with. Over the years I’ve seen far too much of people assuming these skills aren’t valuable, even as people look around and identify folks with these skills as the colleagues they like working with the most. Huge thanks to Kate for bringing attention to these skills. She also talked a lot about building trust within your organization as it can often be hard for managers to do evaluations of employees who have the freedom to work unobstructed (as we want!) and mechanisms to build that trust, including reporting what you do to your boss and team mates. Slides from her talk available here: Keynote: Trust Me slides

After the keynote I headed over to Evan Scheessele’s talk on Infrastructure-as-Code with Puppet Enterprise in the Cloud. He works in HP’s Printing & Personal Systems division and shared the evolution and use of a code-driven infrastructure on HP Cloud along with Puppet Enterprise. The driving vision in his organization was boiled down to a series of points:

  • Infrastructure as “Cattle” not “Pets”
  • Modern configuration-management means: Executable Documentation
  • “Infrastructure as Code”
  • Focus on the production-pattern, and automate it end-to-end
  • Everything is consistently reproducible

He also went application-specific, discussing their use of Jenkins, and hiera and puppetdb in PE. It was a great talk and a pleasure to catch up with him afterwards. Slides available here.


Thanks to Evan Scheessele for the photo

My talk was on How to Open Source Your Puppet Configuration and I brought along Monty Taylor and James E. Blair stick puppets I made to demonstrate the rationale of running our infrastructure as an open source project. I walked the audience through some of the benefits of making Puppet configurations public (or at least public within an organization), the importance of licensing and documentation and some steps for splitting up your configuration so it’s understandable and consumable by others. My slides are here.

On Wednesday I attended Gareth Rushgrove’s talk on Continuous Integration for Infrastructure as Code. He skipped over a lot of the more common individual testing mechanisms (puppet-lint, puppet-syntax, rspec-puppet, beaker) and dove into higher level view things like testing of images and containers and test-driven infrastructure (analogous to test-driven development). Through his talk he gave several examples of how this is accomplished, from use of Serverspec, the need to write tests before infrastructure, writing tests to enforce policy and pulling data from PuppetDB to run tests. Slides here.

After lunch I headed over to Chris Hoge’s talk about Understanding OpenStack Deployments with the Puppet modules available. In spite of all my work with OpenStack, I haven’t had a very close look at these modules, so it was nice meeting up with him and Colleen Murphy from the puppet-openstack team. In his talk he walked the audience through some of the basic design decisions of OpenStack and then pulled in examples of how the Puppet modules for OpenStack are used to bring this all together. Slides here.

Two talks I’ll have to catch once the videos are online, Continuous Infrastructure: Modern Puppet for the Jenkins Project – R.Tyler Croy, Jenkins (slides) and Infrastructure as Software – Dustin J. Mitchell, Mozilla, Inc. (slides). Both of these are open source infrastructures that I mentioned during my own talk! I wish I had taken the opportunity while we were all in one spot to meet together, fortunately I was able to chat with R.Tyler Croy prior to my talk, but his talk conflicted with mine and Dustin’s with the OpenStack talk.

In all, this was a very valuable event. I learned some interesting new things about how others are using code-driven infrastructures and I made some great connections.

More photos from PuppetConf here: https://www.flickr.com/photos/pleia2/sets/72157648049231891/

Offline, CLI-based Gerrit code review with Gertty

This past week I headed to Florida to present at Fossetcon and thought it would be a great opportunity to do a formal review of a new tool recently released by the OpenStack Infrastructure team (well, mostly James E. Blair): Gertty.

The description of this tool is as follows:

As compared to the web interface, the main advantages are:

  • Workflow — the interface is designed to support a workflow similar to reading network news or mail. In particular, it is designed to deal with a large number of review requests across a large number of projects.
  • Offline Use — Gertty syncs information about changes in subscribed projects to a local database and local git repos. All review operations are performed against that database and then synced back to Gerrit.
  • Speed — user actions modify locally cached content and need not wait for server interaction.
  • Convenience — because Gertty downloads all changes to local git repos, a single command instructs it to checkout a change into that repo for detailed examination or testing of larger changes.

For me the two big ones were CLI-based workflow and offline use, I could review patches while on a plane or on terrible hotel wifi!

I highly recommend reading the announcement email to learn more about the features, but to get going here’s a quick rundown for the currently released version 1.0.2:

First, you’ll need to set a password in Gerrit so you can use the REST API. Do that by logging into Gerrit and going to https://review.openstack.org/#/settings/http-password

From there:

pip install gertty

wget https://git.openstack.org/cgit/stackforge/gertty/plain/examples/openstack-gertty.yaml -O ~/.gertty.yaml

Edit ~/.gertty.yaml and update anything that says “CHANGEME”

A couple things worthy of note:

  • Be aware that by default, uses ~/git/ for the git-root, I had to change this in my ~/.gertty.yaml so it didn’t touch my existing ~/git/ directory.
  • You can also run it in a venv, as described on the pypi page.

Now run gertty from your terminal!

When you first load it up, you get a welcome screen with some hints on how to use it, including the all important “press F1 for help”:

Note: I use xfce4-terminal and F1 is bound to terminal help, see the Xfce FAQ to learn how to disable this so you can actually read the Gertty help and don’t have to ask on IRC how to do simple things like I did ;)

As instructed, from here you hit “L” to list projects, this is the page where you can subscribe to them:

You subscribe to projects by pressing “s” and they will show up as bright white, then you can navigate into them to list open reviews:

Go to a review you want to look at and hit enter, bringing up the review screen. This should look very familiar, just text only. I’ve expanded my standard 80×24 terminal window here so you can get a good look at what the full screen looks like:

Navigate down to < Diff > to see the diff. This is pretty cool, instead of showing it on separate pages like the web UI, it shows you a unified page with all of the file diffs, so you just need to scroll through them to see them all:

Finally, you review! Select < Review > back on the main review page and it will pop up a screen that allows you to select your +2, +1, -1, etc and add a message:

Your reviews are synced along with everything else when Gertty knows it’s online and can pull down review updates and upload your changes. At any time you can look at the top right of your screen to see how many pending sync requests it has.

When you want to quit, CTRL-q

I highly recommend giving it a spin. Feel free to ask questions about usage in #openstack-infra and bugs are tracked in Storyboard here: https://storyboard.openstack.org/#!/project/698. The code lives in a stackforge repo at: http://git.openstack.org/cgit/stackforge/gertty

Fossetcon 2014

As I wrote in my last post I attended Fossetcon this past weekend. The core of the event kicked off on Friday with a keynote by Iris Gardner on how Diversity Creates Innovation and the work that the CODE2040 organization is doing to help talented minorities succeed in technology. I first heard about this organization back in 2013 at OSCON, so it was great to hear more about their recent successes with their summer Fellows Program. It was also great to hear that their criteria for talent not only included coding skills, but also sought out a passion for engineering and leadership skills.

After a break, I went to see PJ Hagerty give his talk, Meetup Groups: Act Locally – Think Globally. I’ve been running open source related groups for over a decade, so I’ve been in this space for quite a long time and was hoping to get some new tips, PJ didn’t disappoint! He led off with the need to break out of the small “pizza and a presentation by a regular” grind, which is indeed important to growing a group and making people show up. Some of his suggestions for doing this included:

  • Seek out students to attend and participate in the group, they can be some of your most motivated attendees and will bring friends
  • Seek out experienced programmers (and technologists) not necessarily in your specific field to give more agnostic talks about general programming/tech practices
  • Do cross-technology meetups – a PHP and Ruby night! Maybe Linux and BSD?
  • Bring in guest speakers from out of town (if they’re close enough, many will come for the price of gas and/or train/bus ticket – I would!)
  • Send members to regional conferences… or run your own conference
  • Get kids involved
  • Host an OpenHack event

I’ll have to see what my co-conspiratorsorganizers at some local groups think of these ideas, it certainly would be fun to spice up some of the groups I regularly attend.

From there I went to MySQL Server Performance Tuning 101 by Ligaya Turmelle. Her talk centered around the fact that MySQL tuning is not simple, but went through a variety of mechanisms to tune it in different ways for specific cases you may run into. Perhaps most useful to me were her tips for gathering usage statistics from MySQL, I was unfamiliar with many of the metrics she pulled out. Very cool stuff.

After lunch and some booth duty, I headed over to Crash Course in Open Source Cloud Computing presented by Mark Hinkle. Now, I work on OpenStack (referred to as the “Boy Band” of cloud infrastructures in the talk – hah!), so my view of the cloud world is certainly influenced by that perspective. It was great to see a whirlwind tour of other and related technologies in the open source ecosystem.

The closing keynote for the day was by Deb Nicholson, Style or substance? Free Software is Totally the 80’s. She gave a bit of a history of free software and speculated as to whether our movement would be characterized by a shallow portrayal of “unconferences and penguin swag” (like 80s neon clothes and extravagance) or how free software communities are changing the world (like groups in the 80s who were really seeking social change or the fall of the Berlin wall). Her hope is that by stepping back and taking a look at our community that perhaps we could shape how our movement is remembered and focus on what is important to our future.

Saturday I had more booth duty with my colleague Yolanda Robla who came in from Spain to do a talk on Continuous integration automation. We were joined by another colleague from HP, Mark Atwood, who dropped by the conference for his talk How to Get One of These Awesome Open Source Jobs – one of my favorites.

The opening keynote on Saturday was Considering the Future of Copyleft by Bradley Kuhn. I always enjoy going to his talks because I’m considerably more optimistic about the health and future of free software, so his strong copyleft stance makes me stop and consider where I truly stand and what that means. He worries that an ecosystem of permissive licenses (like Apache, MIT, BSD) will lead to companies doing the least possible for free software and keeping all their secret sauces secret, diluting the ecosystem and making it less valuable for future consumers of free software since they’ll need the proprietary components. I’m more hopeful than that, particularly as I see real free software folks starting to get jobs in major companies and staying true to their free software roots. Indeed, these days I spend a vast majority of my time working on Apache-licensed software for a large company who pays me to do the work. Slides from his talk are here, I highly recommend having a browse: http://ebb.org/bkuhn/talks/FOSSETCON-2014/copyleft-future.html

After some more boothing, I headed over to Apache Mesos and Aurora, An Operating System For The Datacenter by David Lester. Again, being on the OpenStack bandwagon these past few years I haven’t had a lot of time to explore the ecosystem elsewhere, and I learned that this is some pretty cool stuff! Lester works for Twitter and talked some about how Twitter and other companies in the community are using both the Mesos and Aurora tools to build their efficient, fault tolerant datacenters and how it’s lead to impressive improvements in the reliability of their infrastructures. He also did a really great job explaining the concepts of both, hooray for diagrams. I kind of want to play with them now.

Introduction to The ELK Stack: Elasticsearch, Logstash & Kibana by Aaron Mildenstein was my next stop. We run an ELK stack in the OpenStack Infrastructure, but I’ve not been very involved in the management of that, instead focusing on how we’re using it in elastic-recheck so I hoped this talk would fill in some of the fundamentals for me. It did do that so I was happy with that, but I have to admit that I was pretty disappointed to see demos of plugins that required a paid license.

As the day wound down, I finally had my talk: Code Review for Systems Administrators.


Code Review for Sysadmins talk, thanks to Yolanda Robla for taking the photo

I love giving this talk. I’m really proud of the infrastructure that has been built for OpenStack and it’s one that I’m happy and excited to work with every day – in part because we do things through code review. Even better, my excitement during this presentation seemed contagious, with an audience that seemed really engaged with the topic and impressed. Huge thanks to everyone who came and particularly to those who asked questions and took time to chat with me after. Slides from my talk are available here: fossetcon-code-review-for-sysadmins/

And then we were at the end! The conference wrapped up with a closing keynote on Open Source Is More than Code by Jordan Sissel. I really loved this talk. I’ve known for some time that the logstash community was one of the friendlier ones, with their mantra of “If a newbie has a bad time, it’s a bug.” This talk dove further into that ethos in their community and how it’s impacted how members of the project handle unhappy users. He also talked about improvements made to documentation (both inline in code and formal documentation) and how they’ve tried to “break away from text” some and put more human interaction in their community so people don’t feel so isolated and dehumanized by a text only environment (though I do find this is where I’m personally most comfortable, not everyone feels that way). I hope more projects will look to the logstash community as a good example of how we all can do better, I know I have some work to do when it comes to support.

Thanks again to conference staff for making this event such a fun one, particularly as it was their first year!

Ubuntu at Fossetcon 2014

Last week I flew out to the east coast to attend the very first Fossetcon. The conference was on the smaller side, but I had a wonderful time meeting up with some old friends, meeting some new Ubuntu enthusiasts and finally meeting some folks I’ve only communicated with online. The room layout took some getting used to, but the conference staff was quick to put up signs and directing conference attendees in the right direction and in general leading to a pretty smooth conference experience.

On Thursday the conference hosted a “day zero” that had training and an Ubucon. I attended the Ubucon all day, which kicked off with Michael Hall doing an introduction to the Ubuntu on Phones ecosystem, including Mir, Unity8 and the Telephony features that needed to be added to support phones (voice calling, SMS/MMs, Cell data, SIM card management). He also talked about the improved developer portal with more resources aimed at app developers, including the Ubuntu SDK and simplified packaging with click packages.

He also addressed the concern of many about whether Ubuntu could break into the smartphone market at this point, arguing that it’s a rapidly developing and changing market, with every current market leader only having been there for a handful of years, and that new ideas need need to play to win. Canonical feels that convergence between phone and desktop/laptop gives Ubuntu a unique selling point and that users will like it because of intuitive design with lots of swiping and scrolling actions, gives apps the most screen space possible. It was interesting to hear that partners/OEMs can offer operator differentiation as a layer without fragmenting the actual operating system (something that Android struggles with), leaving the core operating system independently maintained.

This was followed up by a more hands on session on Creating your first Ubuntu SDK Application. Attendees downloaded the Ubuntu SDK and Michael walked through the creation of a demo app, using the App Dev School Workshop: Write your first app document.

After lunch, Nicholas Skaggs and I gave a presentation on 10 ways to get involved with Ubuntu today. I had given a “5 ways” talk earlier this year at the SCaLE in Los Angeles, so it was fun to do a longer one with a co-speaker and have his five items added in, along with some other general tips for getting involved with the community. I really love giving this talk, the feedback from attendees throughout the rest of the conference was overwhelmingly positive, and I hope to get some follow-up emails from some new contributors looking to get started. Slides from our presentation are available as pdf here: contributingtoubuntu-fossetcon-2014.pdf


Ubuntu panel, thanks to Chris Crisafulli for the photo

The day wrapped up with an Ubuntu Q&A Panel, which had Michael Hall and Nicholas Skaggs from the Community team at Canonical, Aaron Honeycutt of Kubuntu and myself. Our quartet fielded questions from moderator Alexis Santos of Binpress and the audience, on everything from the Ubuntu phone to challenges of working with such a large community. I ended up drawing from my experience with the Xubuntu community a lot in the panel, especially as we drilled down into discussing how much success we’ve had coordinating the work of the flavors with the rest of Ubuntu.

The next couple days brought Fossetcon proper, with I’ll write about later. The Ubuntu fun continued though! I was able to give away 4 copies of The Official Ubuntu Book, 8th Edition which I signed, and got José Antonio Rey to sign as well since he had joined us for the conference from Peru.

José ended up doing a talk on Automating your service with Juju during the conference, and Michael Hall had the opportunity to a talk on Convergence and the Future of App Development on Ubuntu. The Ubuntu booth also looked great and was one of the most popular of the conference.

I really had a blast talking to Ubuntu community members from Florida, they’re a great and passionate crowd.

Simcoe’s August 2014 Checkup

This upcoming December will mark Simcoe living with the CRF diagnosis for 3 years. We’re happy to say that she continues to do well, with this latest batch of blood work showing more good news about her stable levels.

Unfortunately we brought her in a few weeks early this time following a bloody sneeze. As I’ve written earlier this year, they’ve both been a bit sneezy this year with an as yet undiagnosed issue that has been eluding all tests. Every month or so they switch off who is sneezing, but this was the first time there was any blood.

Simcoe at vet
“I still don’t like vet visits.”

Following the exam, the vet said she wasn’t worried. The bleeding was a one time thing and could have just been caused by rawness brought on by the sneezing and sniffles. Since the appointment on August 26th we haven’t seen any more problems (and the cold seems to have migrated back to Caligula).

As for her levels, it was great to see her weight come up a bit, from 9.62 to 9.94lbs.

Her BUN and CRE levels have both shifted slightly, from 51 to 59 on BUN and 3.9 to 3.8 on CRE.

BUN: 59 (normal range: 14-36)
CRE: 3.8 (normal range: .6-2.4)

CI, Validation and more at DebConf14

I’ve been a Debian user since 2002 and got my first package into Debian in 2006. Though I continued to maintain a couple packages through the years, my open source interests (and career) have expanded significantly so that I now spend much more time with Ubuntu and OpenStack than anything else. Still, I do still host Bay Area Debian events in San Francisco and when I learned that DebConf14 would only be quick plane flight away from home I was eager for the opportunity to attend.

Given my other obligations, I decided to come in halfway through the conference, arriving Wednesday evening. Thursday was particularly interesting to me because they were doing most of the Debian Validation & CI discussions then. Given my day job on the OpenStack Infrastructure team, it seemed to be a great place to meet other folks who are interested in CI and see where our team could support Debian’s initiatives.

First up was the Validation and Continuous Integration BoF led by Neil Williams.

It was interesting to learn the current validation methods being used in Debian, including:

From there talk moved into what kinds of integration tests people wanted, where various ideas were covered, including package sets (collections of related packages) and how to inject “dirty” data into systems to test in more real world like situations. Someone also mentioned doing tests on more real systems rather than in chrooted environments.

Discussion touched upon having a Gerrit-like workflow that had packages submitted for review and testing prior to landing in the archive. This led to my having some interesting conversations with the drivers of Gerrit efforts in Debian after the session (nice to meet you, mika!). There was also discussion about notification to developers when their packages run afoul of the testing infrastructure, either themselves or as part of a dependency chain (who wants notifications? how to make them useful and not overwhelming?).

I’ve uploaded the gobby notes from the session here: validation-bof and the video of the session is available on the meetings-archive.

Next up on the schedule was debci and the Debian Continuous Integration project presented by Antonio Terceiro. He gave a tour of the Debian Continuous Integration system and talked about how packages can take advantage of the system by having their own test suites. He also discussed some about the current architecture for handling tests and optimizations they want to make in the future. Documentation for debci can be found here: ci.debian.net/doc/. Video of the session is also available on the meetings-archive.

The final CI talk I went to of the day was Automated Validation in Debian using LAVA where Neil Williams gave a tour of the expanded LAVA (Linaro Automated Validation Architecture). I heard about it back when it was a more simple ARM-only testing infrastructure, but it’s grown beyond that to now test distribution kernel images, package combinations and installer images and has been encouraging folks to write tests. He also talked about some of the work they’re doing to bring along LAVA demo stations to conferences, nice! Slides from this talk are available on the debconf annex site, here: http://annex.debconf.org/debconf-share/debconf14/slides/lava/

On Friday I also bumped into a testing-related talk by Paul Wise during a series of Live Demos, he showed off check-all-the-things which runs a pile of tools against your project to check… all the things, detecting what it needs to do automatically. Check out the README for rationale, and for a taste of things it checks and future plans, have a peek at some of the data files, like this one.

It’s really exciting to see more effort being spent on testing in Debian, and open source projects in general. This has long been the space of companies doing private, internal testing of open source products they use and reporting results back to projects in the form of patches and bug reports. Having the projects themselves provide QA is a huge step for the maturity of open source, and I believe will lead to even more success for projects as we move into the future.

The rest of DebConf for me was following my more personal interests in Debian. I also have to admit that my lack of involvement lately made me feel like a bit of an outsider and I’m quite shy anyway, so I was thankful to know a few Debian folks who I could hang out with and join for meals.

On Thursday evening I attended A glimpse into a systemd future by Josh Triplett. I haven’t really been keeping up with systemd news or features, so I learned a lot. I have to say, it would be great to see things like session management, screen brightness and other user settings be controlled by something lower level than the desktop environment. Friday I attended Thomas Goirand’s OpenStack update & packaging experience sharing. I’ve been loosely tracking this, but it was good to learn that Jessie will come with Icehouse and that install docs exist for Wheezy (here).

I also attended Outsourcing your webapp maintenance to Debian with Francois Marier. The rationale for his talk was that one should build their application with the mature versions of web frameworks included with Debian in mind, making it so you don’t have the burden of, say, managing Django along with your Django-based app, since Debian handles that. I continue to have mixed feelings when it comes to webapps in the main Debian repository, while some developers who are interested in reducing maintenance burden are ok with using older versions shipped with Debian, most developers I’ve worked with are very much not in this camp and I’m better off trying to support what they want than fighting with them about versions. Then it was off to Docker + Debian = ♥ with Paul Tagliamonte where he talked about some of his best practices for using Docker on Debian and ideas for leveraging it more in development (having multiple versions of services running on one host, exporting docker images to help with replication of tests and development environments).

Friday night Linus Torvalds joined us for a Q&A session. As someone who has put a lot of work into making friendly environments for new open source contributors, I can’t say I’m thrilled with his abrasive conduct in the Linux kernel project. I do worry that he sets a tone that impressionable kernel hackers then go on to emulate, perpetuating the caustic environment that spills out beyond just the kernel, but he has no interest in changing. That aside, it was interesting to hear him talk about other aspects of his work, his thoughts on systemd, a rant about compiling against specific libraries for every distro and versions (companies won’t do it, they’ll just ship their own statically linked ones) and his continued comments in support of Google Chrome.

DebConf wrapped up on Sunday. I spent the morning in one of the HackLabs catching up on some work, and at 1:30 headed up to the Plenary room for the last few talks of the event, starting with a series of lightning talks. A few of the talks stood out for me, including Geoffrey Thomas’ talk on being a bit of an outsider at DebConf and how difficult it is to be running a non-Debian/Linux system at the event. I’ve long been disappointed when people bring along their proprietary OSes to Linux events, but he made good points about people being able to contribute without fully “buying in” to having free software everywhere, including their laptop. He’s right. Margarita Manterola shared some stats from the Mini-DebConf Barcelona which focused on having only female speakers, it was great to hear such positive statistics, particularly since DebConf14 itself had a pretty poor ratio, there were several talks I attended (particularly around CI) where I was the only woman in the room. It was also interesting to learn about safe-rm to save us from ourselves and non-free.org to help make a distinction between what is Debian and what is not.

There was also a great talk by Vagrant Cascadian about his work on packages that he saw needed help but he didn’t necessarily know everything about, and encouraged others to take the same leap to work on things that may be outside their comfort zone. To help he listed several resources people could use to find work in Debian:

Next up for the afternoon was the Bits from the Release Team where they fleshed out what the next few months leading up to the freeze would look like and sharing the Jessie Freeze Policy.

DebConf wrapped up with a thank you to the volunteers (thank you!) and peek at the next DebConf, to be held in Heidelberg, Germany the 15th-22nd of August 2015.

Then it was off to the airport for me!

The rest of my photos from DebConf14 here: https://www.flickr.com/photos/pleia2/sets/72157646626186269/