• Archives

  • Categories

  • Other profiles


  • wallaceadngromit.net

  • Partimus

  • Secular Humanism

  • Debian

  • Ubuntu Women

  • Xubuntu

  • OpenStack

Liberty OpenStack Summit days 3-5

Summiting continued! The final three days of the conference offered two days of OpenStack Design Summit discussions and working sessions on specific topics, and Friday was spent doing a contributors meetup so we could have face time with people we’re working with on projects.

Wednesday began with a team breakfast, where over 30 of us descended upon a breakfast restaurant and had a lively morning. Unfortunately it ran a bit long and made us a bit late for the beginning of summit stuff, but the next Infrastructure work session was fully attended! The session sought to take some next steps with our activity tracking mechanisms, none of which are currently part of the OpenStack Infrastructure. Currently there are several different types of stats being collected, from reviewstats which are hosted by a community member and focus specifically on reviews to those produced from Bitergia (here) that are somewhat generic but help compare OpenStack to other open source projects to Stackalytics which is crafted specifically for the OpenStack community. There seems to be value in hosting various metric types, mostly so comparisons can be made across platforms if they differ in any way. The consensus of the session was to first move forward with moving Stackalytics into our infrastructure, since so many projects find such value in it. Etherpad here: YVR-infra-activity-tracking


With this view from the work session room, it’s amazing we got anything done

Next up was QA: Testing Beyond the Gate. In OpenStack there is a test gate that all changes must pass in order for a change to be merged. In the past cycle periodic and post-merge tests have also been added, but it’s been found that if a code merging isn’t dependent upon these passing, not many people pay attention to these additional tests. The result of the session is a proposed dashboard for tracking these tests so that there’s an easier view into what they’re doing, whether they’re failing and so empower developers to fix them up. Tracking of third party testing in this, or a similar, tracker was also discussed as a proposal once the infra-run tests are being accounted for. Etherpad here: YVR-QA-testing-beyond-the-gate

The QA: DevStack Roadmap session covered some of the general cleanup that typically needs to be done in DevStack, but then also went into some of the broader action items, including improving the reliability of Centos tests run against it that are currently non-voting, pulling some things out of DevStack to support them as plugins as we move into a Big Tent world and work out how to move forward with Grenade. Etherpad here: YVR-QA-Devstack-Roadmap

I then attended QA: QA in the Big Tent. In the past cycle, OpenStack dropped the long process of being accepted into OpenStack as an official project and streamlined it so that competing technologies are now all in the mix, we’re calling it the Big Tent – as we’re now including everyone. This session focused on how to support the QA needs now that OpenStack is not just a slim core of a few projects. The general idea from a QA perspective is that they can continue to support the things-everyone-uses (nova, neutron, glance… an organically evolving list) and improve pluggable support for projects beyond that so they can help themselves to the QA tools at their disposal. Etherpad here: YVR-QA-in-the-big-tent

With sessions behind me, I boarded a bus for the Core Reviewer Party, hosted at the Museum of Anthropology at UBC. As party venues go, this was a great one. The museum was open for us to explore, and they also offered tours. The main event took place outside, where they served design-your-own curry seafood dishes, bison, cheeses and salmon. Of course no OpenStack event would be complete with a few bars around serving various wines and beer. There was an adjacent small building where live music was playing and there was a lot of space to walk around, catch the sunset and enjoy some gardens. I spent much of my early evening with friends from Time Warner Cable, and rounded things off with several of my buddies from HP. This ended up being a get-back-after-midnight event for me, but it was totally worth it to spend such a great time with everyone.

Thursday morning kicked off with a series of fishbowl sessions where the Infrastructure team was discussing projects we have in the works. First up was Infrastructure: Zuul v3. Zuul is our pipeline-oriented project gating system, which currently works by facilitating the of running tests and automated tasks in response to Gerrit events. Right now it sends jobs off to Gearman for launching via Jenkins to our fleet of waiting nodes, but we’re really using Jenkins as a shim here, not really taking advantage of the built in features that Jenkins offers. We’re also in need of a system that better supports multi-tenancy and multi-node jobs and which can scale as OpenStack continues to grow, particularly with Big Tent. This session discussed the end game of phasing out Jenkins in favor of a more Zuul-driven workflow and more immediate changes that may be made to Nodepool and smaller projects like Zuul-merger to drive our vision. Etherpad here: YVR-infra-zuulv3

Everyone loves bug reporting and task tracking, right? In the next session, Infrastructure: Task tracking, that was our topic. We did an experiment with the creation of Storyboard as our homebrewed solution to bug and task tracking, but in spite of valiant efforts by the small team working on it, they were unable to gain more contributors and the job was simply too big for the size of the team doing the work. As a result, we’re now back to looking at solutions other than Canonical’s hosted Launchpad (which is currently used). The session went through some basic evaluation of a few tools, and at the end there was some consensus to work toward bringing up a more battle-hardened and Puppetized instance of Maniphest (from Phabricator) so that teams can see if it fits their needs. Etherpad here:YVR-infra-task-tracking

The morning continued with an Infrastructure: Infra-cloud session. The Infrastructure team has about 150 machines in a datacenter that have been donated to us by HP. The session focused on how we can put these to use as Nodepool instances by running OpenStack on our own and adding that “infra-cloud” to the providers in Nodepool. I’m particularly interested in this, given some of my history with getting TripleO into testing (so have deployed OpenStack many, many times!) and in general eager to learn even more about production OpenStack deployments. So it looks like I’ll be providing Infra-brains to Clint Byrum who is otherwise taking a lead here. To keep in sync with other things we host, we’ll be using Puppet to deploy OpenStack, so I’m thankful for the expertise of people like Colleen Murphy who just joined our team to help with that. Etherpad here: YVR-infra-cloud

Next up was the Infrastructure: Puppet testing session. It was great to have some of the OpenStack Puppet folks in the room so they could talk some about how they’re using beaker-rspec in our infra for testing the OpenStack modules themselves. Much of the discussion centered around whether we want to follow their lead, or do something else, leveraging our current system of node allocation to do our own module testing. We also have a much commented on spec up for proposal here. The result of the discussion was that it’s likely that we’ll just follow the lead of the OpenStack Puppet team. Etherpad here: kilo-infra-puppet-testing

That afternoon we had another Infrastructure: Work session where we focused on the refactor of portions of system-config OpenStack module puppet scripts, and some folks worked on getting the testing infrastructure that was talked about earlier. I took the opportunity to do some reviews of the related patches and help a new contributor do some review – she even submitted a patch that was merged the next morning! Etherpad for the work session here: YVR-infra-puppet-openstackci

The last session I attended that day was QA: Liberty Priorities. It wasn’t one I strictly needed to be in, but I hadn’t attended a session in room 306 yet, and it was the famous gosling room! The room had a glass wall that looked out onto a roof were a couple of geese had their babies and would routinely walk by and interrupt the session because everyone would stop, coo and take pictures of them. So I finally got to see the babies! The actual session collected the pile of to do list items generated at the summit, which I got roped into helping with, and prioritized them. Oh, and they gave me a task to help with. I just wanted to see the geese! Etherpad with the priorities is here: YVR-QA-Liberty-Priorities


Photo by Thierry Carrez (source)

Thursday night I ended up having dinner with the moderator of our women of OpenStack panel, Beth Cohen. We went down to Gastown to enjoy a dinner of oysters and seafood and had a wonderful time. It was great to swap tech (and women in tech) stories and chat about our work.

Friday! The OpenStack conference itself ended on Thursday, so it was just ATCs (Active Technical Contributors) attending for the final day of the Design Summit. So things were much quieter and the agenda was full of contributors meetups. I spent the day in the Infrastructure, QA and Release management contributors meetup. We had a long list of things to work on, but I focused on the election tooling, which I ended up following up with on list and then later had a chat with the author of the proposed tooling. My afternoon was spent working on the translations infrastructure with Steve Kowalik who works with me on OpenStack infra and Carlos Munoz of the Zanata team. We were able to work through the outstanding Zanata bugs and make some progress with how we’re going to tackle everything, it was a productive afternoon and always a pleasure to get together with the folks I work with online every day.

That evening, as we left the closing conference center, I met up with several colleagues for an amazing sushi dinner in downtown Vancouver. A perfect, low-key ending to an amazing event!

Liberty OpenStack Summit day 2

My second day of the OpenStack summit came early with he Women of OpenStack working breakfast at 7AM. It kicked off with a series of lightning talks that talked about impostor syndrome, growing as a technical leader (get yourself out there, ask questions) and suggestions from a tech start-up founder about being an entrepreneur. From there we broke up into groups to discuss what we’d like to see from the Women of OpenStack group in the next year. The big take-aways were around mentoring of new women joining our community and starting to get involved with all the OpenStack tooling and more generally giving voice to the women in our community.

Keynotes kicked off at 9AM with Mark Collier announcing the next OpenStack Summit venues: Austin for the spring 2016 summit and Barcelona for the fall 2016 summit. He then went into a series of chats and demos related to using containers, which may be the Next Big Thing in cloud computing. During the session we heard from a few companies who are already using OpenStack with containers (mostly Docker and Kubernetes) in production (video). The keynotes continued with one by Intel, where the speaker took time to talk about how valuable feedback from operators has been in the past year, and appreciation for the new diversity working group (video). The keynote from EBay/Paypal showed off the really amazing progress they’ve made with deploying OpenStack, with it now running on over 300k cores and pretty much powers Paypal at this point (video). Red Hat’s keynote focused on customer engagement as OpenStack matures (video). The keynotes wrapped up with one from NASA JPL, which mostly talked about the awesome Mars projects they’re working on and the massive data requirements therein (video).


OpenStack at EBay/Paypal

Following keynotes, Tuesday really kicked off the core OpenStack Design Summit sessions, where I focused on a series of Cross Project Workshops. First up was Moving our applications to Python 3. This session focused on the migration of Python 3 for functional and integration testing in OpenStack projects now that Oslo libraries are working in Python 3. The session mostly centered around strategy, how to incrementally move projects over and the requirements for the move (2.x dependencies, changes to Ubuntu required to effectively use Python 3.4 for gating, etc). Etherpad here: liberty-cross-project-python3. I then attended Functional Testing Show & Tell which was a great session where projects shared their stories about how they do functional (and some unit) testing in their projects. The Etherpad for this one is super valuable for seeing what everyone reports, it’s available here: liberty-functional-testing-show-tell.

My Design Summit sessions were broken up nicely with a lunch with my fellow panelists, and then the Standing Tall in the Room – Sponsored by the Women of OpenStack panel itself at 2PM (video). It was wonderful to finally meet my fellow panelists in person and the session itself was well-attended and we got a lot of positive feedback from it. I tackled a question about shyness with regard to giving presentations here at the OpenStack Summit, where I pointed at a webinar about submitting a proposal via the Women of OpenStack published in January. I also talked about difficulties related to the first time you write to the development mailing list, participate on IRC and submit code for review. I used an example of having to submit 28 patches for one of my early patches, and audience member Steve Martinelli helpfully tweeted about a 63 patch change. Diving in to all these things helps, as does supporting the ideas of and doing code review for others in your community. Of course my fellow panelists had great things to say too, watch the video!


Thanks to Lisa-Marie Namphy for the photo!

Panel selfie by Rainya Mosher

Following the panel, it was back to the Design Summit. The In-team scaling session was an interesting one with regard to metrics. We’ve learned that regardless of project size, socially within OpenStack it seems difficult for any projects to rise above 14 core reviewers, and keep enough common culture, focus and quality. The solutions presented during the session tended to be heavy on technology (changes to ACLs, splitting up the repo to trusted sub-groups). It’ll be interesting to see how the scaling actually pans out, as there seem to be many more social and leadership solutions to the problem of patches piling up and not having enough core folks to review them. There was also some discussion about the specs process, but the problems and solutions seem to heavily vary between teams, so it seemed unlikely that a unified solution to unprocessed specs would be universal, but it does seem like the process is often valuable for certain things. Etherpad here: liberty-cross-project-in-team-scaling.

My last session of the day was OpenStack release model(s). A time-based discussion required broader participation, so much of the discussion centered around the ability for projects to independently do intermediary releases outside of the release cycle and how that could be supported, but I think the jury is still out on a solution there. There was also talk about how to generally handle release tracking, as it’s difficult to predict what will land, so much so that people have stopped relying on the predictions and that bled into a discussion about release content reporting (release changelogs). In all, an interesting session with some good ideas about how to move forward, Etherpad here: liberty-cross-project-release-models.

I spent the evening with friends and colleagues at the HP+Scality hosted party at Rocky Mountaineer Station. BBQ, food trucks and getting to see non-Americans/non-Canadians try s’mores for the first time, all kinds of fun! Fortunately I managed to make it back to my hotel at a reasonable hour.

Liberty OpenStack Summit day 1

This week I’m at the OpenStack Summit. It’s the most wonderful, exhausting and valuable-to-my-job event I go to, and it happens twice a year. This time it’s being held in the beautiful city of Vancouver, BC, and the conference venue is right on the water, so we get to enjoy astonishing views throughout the day.


OpenStack Summit: Clouds inside and outside!

Jonathan Bryce Executive Director of the OpenStack Foundation kicked off the event with an introduction to the summit, success that OpenStack has built in the Process, Store and Move digital economy, and some announcements, among which was the success found with federated identity support in Keystone where Morgan Fainberg, PTL of Keystone, helped show off a demonstration. The first company keynote was presented by Digitalfilm Tree who did a really fun live demo of shooting video at the summit here in Vancouver, using their OpenStack-powered cloud so it was accessible in Los Angeles for editorial review and then retrieving and playing the resulting video. They shared that a recent show that was shot in Vancouver used this very process for the daily editing and that they had previously used courier services and staff-hopping-on-planes to do the physical moving of digital content because it was too much for their previous systems. Finally, Comcast employees rolled onto the stage on a couch to chat about how they’ve expanded their use of OpenStack since presenting at the summit in Portland, Oregon Video of the all of this available here.

Next up for keynotes was Walmart, who talked about how they moved to OpenStack and used it for all the load on their sites experienced over the 2014 holiday season and how OpenStack has met their needs, video here. Then came HP’s keynote, which really focused on the community and choices available aspect of OpenStack, where speaker Mark Interrante said “OpenStack should be simpler, you shouldn’t need a PhD to run it.” Bravo! He also pointed out that HP’s booth had a demonstration of OpenStack running on various hardware at the booth, an impressively inclusive step for a company that also sells hardware. Video for HP’s keynote here (I dig the Star Wars reference). Keynotes continued with one from TD Bank, which I became familiar with when they bought up the Commerce branches in the Philadelphia region, but have since learned are a major Canadian Bank (oooh, TD stands for Toronto Dominion!). The most fascinating thing about their moved to the cloud for me is how they’ve imposed a cloud-first policy across their infrastructure, where teams must have a really good reason and approval in order to do more traditional bare-metal, one off deployments for their applications, so it’s rare, video. Cybera was the next keynote and perhaps the most inspiring from a humanitarian standpoint. As one of the earliest OpenStack adopters, Cybera is a non-profit that seeks to improve access to the internet and valuable resources therein, which presented Robin Winsor stressed in his keynote was now as the physical infrastructure that was built in North America in the 19th and 20th centuries (railroads, highways, etc), video here. The final keynote was from Solidfire who discussed the importance of solid storage as a basis of a successful deployment, video here.

Following the keynotes, I headed over to the Virtual Networking in OpenStack: Neutron 101 (video) where Kyle Mestery and Mark McClain gave a great overview of how Neutron works with various diagrams showing of the agents and improvements made in Kilo with various new drivers and plugins. The video is well worth the watch.

A chunk of my day was then reserved for translations. My role here is as the Infrastructure team contact for the translations tooling, so it’s also been a crash course in learning about translations workflows since I only speak English. Each session, even unrelated to the actual infrastructure-focused tooling has been valuable to learning. In the first translation team working session the focus was translations glossaries, which are used to help give context/meaning to certain English words where the meaning can be unclear or otherwise needs to be defined in terms of the project. There was representation from the Documentation team, which was valuable as they maintain a docs-focused glossary (here) which is more maintained and has a bigger team than the proposed separate translations glossary would have. Interesting discussion, particularly as my knowledge of translations glossaries was limited. Etherpad here: Vancouver-I18n-WG-session.

I hosted the afternoon session on Building Translation Platform. We’re migrating the team to Zanata have been fortunate to have Carlos Munoz, one of the developers on Zanata, join us at every summit since Atlanta. They’ve been one of the most supportive upstreams I’ve ever worked with, prioritizing our bug reports and really working with us to make sure our adoption is a success. The session itself reviewed the progress of our migration and set some deadlines for having translators begin the testing/feedback cycle. We also talked about hosting a Horizon instance in infra, refreshed daily, so that translators can actually see where translations are most needed via the UI and can prioritize appropriately. Finally, it was a great opportunity to get feedback from translators about what they need from the new workflow and have Carlos there to answer questions and help prioritize bugs. Etherpad here: Vancouver-I18n-Translation-platform-session.

My last translations-related thing of the day was Here be dragons – Translating OpenStack (slides). This was a great talk by Łukasz Jernaś that began with some benefits of translations work and then went into best practices and tips for working with open source translations and OpenStack specifically. It was another valuable session for me as the tooling contact because it gave me insight into some of the pain points and how appropriate it would be to address these with tooling vs. social changes to translations workflows.

From there I went back to general talks, attending Building Clouds with OpenStack Puppet Modules by Emilien Macchi, Mike Dorman and Matt Fischer (video). The OpenStack Infrastructure team is looking at building our own infra-cloud (we have a session on it later this week) and the workflows and tips that this presentation gave would also be helpful to me in other work I’ve been focusing on.

The final session I wandered into was a series of Lightning Talks, put together by HP. They had a great lineup of speakers from various companies and organizations. My evening was then spent at an HP employee gathering, but given my energy level and planned attendance at the Women of OpenStack breakfast at 7AM the following morning I headed back to my hotel around 9PM.

Xubuntu sweatshirt, Wily, & Debian Jessie Release

People like shirts, stickers and goodies to show support of their favorite operation system, and though the Xubuntu project has been slower than our friends over at Kubuntu at offering them, we now have a decent line-up offered by companies we’re friendly with. Several months ago the Xubuntu team was contacted by Gabor Kum of HELLOTUX to see if we’d be interested in offering shirts through their site. We were indeed interested! So after he graciously sent our project lead a polo shirt to evaluate, we agreed to start offering his products on our site, alongside the others. See all products here.

Polos aren’t really my thing, so when the Xubuntu shirts went live I ordered the Xubuntu sweater. Now a language difference may be in play here, since I’d call it a sweatshirt with a zipper, or a light jacket, or a hoodie without a hood. But it’s a great shirt, I’ve been wearing it regularly since I got it in my often-chilly city of San Francisco. It fits wonderfully and the embroidery is top notch.

Xubuntu sweatshirt
Close-up of HELLOTUX Xubuntu embroidery

In other Ubuntu things, given my travel schedule Peter Ganthavorn has started hosting some of the San Francisco Ubuntu Hours. He hosted one last month that I wasn’t available for, and then another this week which I did attend. Wearing my trusty new Xubuntu sweatshirt, I also brought along my Wily Werewolf to his first Ubuntu Hour! I picked up this fluffy-yet-fearsome werewolf from Squishable.com, which is also where I found my Natty Narwhal.

When we wrapped up the Ubuntu Hour, we headed down the street to our favorite Chinese place for Linux meetings where I was hosting a Bay Area Debian Meeting and Jessie Release Party! I was pretty excited about doing this, since the Toy Story character Jessie is a popular one, I jumped at the opportunity to pick up some party supplies to mark the occasion, and ended up with a collection of party hats and notepads:

There were a total of 5 of us there, long time BAD member Michael Paoli being particularly generous with his support of my ridiculous hats:

We had a fun time, welcoming a couple of new folks to our meeting as well. A few more photos from the evening here: https://www.flickr.com/photos/pleia2/sets/72157650542082473

Now I just need to actually upgrade my servers to Jessie!

OpenStack events, anniversary & organization, a museum and some computers & cats

I’ve been home for just over 3 weeks. I thought things would be quieter event-wise, but I have attended 2 OpenStack meetups since getting home, the first right after getting off the plane from South Carolina. My colleague and Keystone PTL Morgan Fainberg was giving a presentation on Keystone and I have the rare opportunity to finally meet a scholarship winner who I’ve been mentoring at work. It was great to meet up and see some of the folks who I only see at conference, including other colleagues from HP. Plus, Morgan’s presentation on Keystone was great and the audience had a lot of good questions. Video of the presentation here and slides are available here


With my Helion mentee!

This past week I went to the second meetup, this time over at Walmart Labs, just a quick walk from the Sunnyvale Caltrain station. For this meetup I was on a mainstage panel where discussions covered improvements to OpenStack in the Kilo release (including the continued rise of third party testing, which I was able to speak to), the new Big Tent approach to OpenStack project adoption and how baremetal is starting to change the OpenStack landscape. I was also able to meet some of the really smart people working at Walmart Labs, and learned that all of walmart.com is running on top of OpenStack (this article from March talks about it and they’ll be doing a session on it at the upcoming OpenStack Summit in Vancouver).


Panel at Walmart Labs

In other professional news, the work I did in Oman earlier this year continues to bear fruit. On April 20th issue #313 of the Sultan Qaboos University Horizon newsletter was published with my interview, (8M PDF here). They were kind enough to send me a few paper copies which I received on Friday. The interview touched upon key points that I spoke on during my presentation back in February, focusing on personal and business reasons for open source contributions.

Personally, MJ and I celebrated our second wedding anniversary with a fantastic meal at Murray Circle Restaurant where we sat on the porch and enjoyed our dinner with a nighttime view of the Golden Gate Bridge. We also recently agreed to start a diet together, largely going back to our pre-wedding diet that we both managed to lose a lot of weight on. Health-wise I continue to go out running, but running isn’t enough to help me to lose weight. I’m largely replacing starches with vegetables and reducing the sugar in my diet. Finally, we’ve been hacking our way through a massive joint to do list that’s been haunting us for several months now. Most of the tasks are home-based, from things like painting we need to get done to storage clean-outs. I don’t love that we have so much to do (don’t other adults get to have fun on weekends?), but finally having it organized and a plan for tackling it has reduced my stress incredibly.


2nd anniversary dinner

We do actually get to have fun on weekends, Saturday at least. We’ve continued to take Saturdays off together to attend services, have a nice lunch together and spend some time relaxing, whether that’s catching up on some shows together or visiting a local museum. Last weekend we had the opportunity of finally going to the Cable Car Museum here in San Francisco. Given my love for all things rail, it’s astonishing that I never made it up there before. The core of the museum is the above-ground, in-building housing for the four cables that run the three cable car lines, and then exhibits are built around it. It’s a fantastic little museum, and entrance is free.

I also picked up some beautifully 3d printed cable car earrings and matching necklace produced by Freeform Ind. I loved their stuff so much that I found their shop online and picked up some other local landmark jewelry.

More photos from our trip to the Cable Car Museum are available here: https://www.flickr.com/photos/pleia2/sets/72157652325687332

We’ve had some computer fun lately. MJ has finally ordered a replacement 1U server for the old one that he has co-located in Fremont. Burn-in testing happened this weekend but there are some more harddrive-related pieces that we’re still waiting on to get it finished up. We’re aiming for getting it installed at the datacenter in June. I also replaced the old Pentium 4 that I’ve been using as a monitoring server and backups machine. It was getting quite old and unusable as a second desktop, even when restricted to following social media accounts and watching videos here and there. It’s now been replaced with a refurbished HP DC6200 from 2011, which has an i3 processor and I bumped it up to 8G of RAM that I had laying around from when I maxed out my primary desktop with 16G. So far so good, I moved over the harddrive from the old machine and it’s been running great.


HP DC6200

In the time between work and other things, I’ve been watching The Good Wife on my own and Star Trek: Voyager with MJ. Also, hanging out with my darling kitties. One evening I got this epic picture of Caligula:

This week I’m hosting an Ubuntu Hour and Debian Dinner where we’re celebrate the release of Debian 8 “Jessie”. I’ve purchased Jessie (cowgirl from Toy Story 2 and 3) party hats to mark the occasion. At the break of dawn on Sunday I’ll be boarding a plane to go to the OpenStack Summit in Vancouver. I’ve never been to Vancouver, so I’m spending Sunday there and staying until late on the following Saturday night, so I hope to have time to see some of the city. After this trip, I’m staying home until July! Thank goodness, I can definitely use the down time to work on my book.

POSSCON 2015

This past week I had the pleasure of attending POSSCON in the beautiful capital city of South Carolina, Columbia. The great event kicked off with a social at Hickory Tavern, which I arranged to be at by tolerating a tight connection in Charlotte. It all worked out and in spite of generally being really shy at these kind of socials, I found some folks I knew and had a good time. Late in the evening several of us even had the opportunity to meet the Mayor of Columbia who had come down to the event and talk about our work and the importance of open source in the economy today. It’s really great to see that kind of support for open source in a city.

The next morning the conference actually kicked off. Organizer Todd Lewis opened the event and quickly handed things off to Lonnie Emard, the President of IT-oLogy. IT-oLogy is a non-profit that promotes initial and continued learning in technology through events targeting everyone from children in grade school to professionals who are seeking to extend their skill set, more on their About page. As a partner for POSSCON, they were a huge part of the event, even hosting the second day at their offices.

We then heard from aforementioned Columbia Mayor Steve Benjamin. A keynote from the city mayor was real treat, taking time out of what I’m sure is a busy schedule showed a clear commitment to building technology in Columbia. It was really inspiring to hear him talk about Columbia, with political support and work from IT-oLogy it sounds like an interesting place to be for building or growing a career in tech. There was then a welcome from Amy Love, the South Carolina Department of Commerce Innovation Director. Talk about local support! Go South Carolina!

The next keynote was from Andy Hunt, who was speaking on “A New Look at Openness” which began with a history of how we’ve progressed with development, from paying for licenses and compilers for proprietary development to the free and open source tool set and their respective licenses we work with today. He talked about how this all progresses into the Internet of Things, where we can now build physical objects and track everything from keys to pets. Today’s world for developers, he argued, is not about inventing but innovating, and he implored the audience to seek out this innovation by using the building blocks of open source as a foundation. In the idea space he proposed 5 steps for innovative thinking:

  1. Gather raw material
  2. Work it
  3. Forget the whole thing
  4. Eureka/My that’s peculiar
  5. Refine and develop
  6. profit!

Directly following the keynote I gave my talk on Tools for Open Source Systems Administration in the Operations/Back End track. It had the themes of many of my previous talks on how the OpenStack Infrastructure team does systems administration in an open source way, but I refocused this talk to be directly about the tools we use to accomplish this as a geographically distributed team across several different companies. The talk went well and I had a great audience, huge thanks to everyone who came out for it, it was a real pleasure to talk with folks throughout the rest of the conference who had questions about specific parts of how we collaborate. Slides from my presentation are here (pdf).

The next talk in the Operations/Back End track was Converged Infrastructure with Sanoid by Jim Salter. With SANOID, he was seeking to bring enterprise-level predictability, minimal downtime and rapid recover to small-to-medium-sized businesses. Using commodity components, from hardware through software, he’s built a system that virtualizes all services and runs on ZFS for Linux to take hourly (by default) snapshots of running systems. When something goes wrong, from a bad upgrade to a LAN infected with a virus, he has the ability to quickly roll users back to the latest snapshot. It also has a system for easily creating on and off-site backups and uses Nagios for monitoring, which is how I learned about aNag, a Nagios client for Android, I’ll have to check it out! I had the opportunity to spend more time with Jim as the conference went on, which included swinging by his booth for a SANOID demo. Slides from his presentation are here.

For lunch they served BBQ. I don’t really care for typical red BBQ sauce, so when I saw a yellow sauce option at the buffet I covered my chicken in that instead. I had discovered South Carolina Mustard BBQ sauce. Amazing stuff. Changed my life. I want more.

After lunch I went to see a talk by Isaac Christofferson on Assembling an Open Source Toolchain to Manage Public, Private and Hybrid Cloud Deployments. With a focus on automation, standardization and repeatability, he walked us through his usage of Packer, Vagrant and Ansible to interface with a variety of different clouds and VMs. I’m also apparently the last systems administrator alive who hadn’t heard of devopsbookmarks.com, but he shared the link and it’s a great site.

The rooms for the talks were spread around a very walkable area in downtown Columbia. I wasn’t sure how I’d feel about this and worried it would be a problem, but with speakers staying on schedule we were afforded a full 15 minutes between talks to switch tracks. The venue I spoke it was in a Hilton, and the next talk I went to was in a bar! It made for quite the enjoyable short walks outside between talks and a diversity in venues that was a lot of fun.

That next talk I went to was Open Source and the Internet of Things presented by Erica Stanley. I had the pleasure of being on a panel with Erica back in October during All Things Open (see here for a great panel recap) so it was really great running into her at this conference as well. Her talk was a deluge of information about the Internet of Things (IoT) and how we can all be makers for it! She went into detail about the technology and ideas behind all kinds of devices, and on slides 41 and 42 she gave a quick tour of hardware and software tools that can be used to build for the IoT. She also went through some of the philosophy, guidelines and challenges for IoT development. Slides from her talk are online here, the wealth of knowledge packed into that slidedeck are definitely worth spending some time with if you’re interested in the topic.

The last pre-keynote talk I went to was by Tarus Balog with a Guide to the Open Source Desktop. A self-confessed former Apple fanboy, he had quite the sense of humor about his past where “everything was white and had an apple on it” and his move to using only open source software. As someone who has been using Linux and friends for almost a decade and a half, I wasn’t at this talk to learn about the tools available, but instead see how a long time Mac user could actually make the transition. It’s also interesting to me as a member of the Ubuntu and Xubuntu projects to see how newcomers view entrance into the world of Linux and how they evaluate and select tools. He walked the audience through the process he used to select a distro and desktop environment and then all the applications: mail, calendar, office suite and more. Of particular interest he showed a preference for Banshee (reminded him of old iTunes), as well as digiKam for managing photos. Accounting-wise he is still tied to Quickbooks, but either runs it under wine or over VNC from a Mac.

The day wound down with a keynote from Jason Hibbets. He wrote The foundation for an open source city and is a Project Manager for opensource.com. His keynote was all about stories, and why it’s important to tell our open source stories. I’ve really been impressed with the development of opensource.com over the past year (disclaimer: I’ve written for them too), they’ve managed to find hundreds of inspirational and beneficial stories of open source adoption from around the world. In this talk he highlighted a few of these, including the work of my friend Charlie Reisinger at Penn Manor and Stu Keroff with students in the Asian Penguins computer club (check out a video from them here). How exciting! The evening wrapped up with an afterparty (I enjoyed a nice Palmetto Amber Ale) and a great speakers and sponsors dinner, huge thanks to the conference staff for putting on such a great event and making us feel so welcome.

The second day of the conference took place across the street from the South Carolina State House at the IT-oLogoy office. The day consisted of workshops, so the sessions were much longer and more involved. But the day also kicked off with a keynote by Bradley Kuhn, who gave a basic level talk on Free Software Licensing: Software Freedom Licensing: What You Must Know. He did a great job offering a balanced view of the licenses available and the importance of selecting one appropriate to your project and team from the beginning.

After the keynote I headed upstairs to learn about OpenNMS from Tarus Balog. I love monitoring, but as a systems administrator and not a network administrator, I’ve mostly been using service-based monitoring tooling and hadn’t really looking into OpenNMS. The workshop was an excellent tour of the basics of the project, including a short history and their current work. He walked us through the basic installation and setup, and some of the configuration changes needed for SNMP and XML-based changes made to various other parts of the infrastructure. He also talked about static and auto-discovery mechanisms for a network, how events and alarms work and details about setting up the notification system effectively. He wrapped up by showing off some interesting graphs and other visualizations that they’re working to bring into the system for individuals in your organization who prefer to see the data presented in less technical format.

The afternoon workshop I attended was put on by Jim Salter and went over Backing up Android using Open Source technologies. This workshop focused on backing up content and not the Android OS itself, but happily for me, that’s what I wanted to back up, as I run stock Android from Google otherwise (easy to install again from a generic source as needed). Now, Google will happily backup all your data, but what if you want to back it up locally and store it on your own system? By using rsync backup for Android, Jim demonstrated how to configure your phone to send backups to Linux, Windows and Mac using ssh+rsync. For Linux at least so far this is a fully open source solution, which I quite like and have started using it at home. The next component makes it automatic, which is where we get into a proprietary bit of software, Llama – Location Profiles. Based on various types of criteria (battery level, location, time, and lots more), Llama allows you to identify criteria of when it runs certain actions, like automatically running rsync to do backups. In all, it was a great and informative workshop and I’m happy to finally have a useful solution to pulling photos and things off my phone periodically without plugging it in and using MTP, which apparently I hate and so never I do it. Slides from Jim’s talk, which also include specific instructions and tools for Windows and Mac are online here.

The conference concluded with Todd Lewis sending more thanks all around. By this time in the day rain was coming down in buckets and there were no taxis to be seen, so I grabbed a ride from Aaron Crosman who I was happy to learn earlier was a local but had come from Philadelphia and we had great Philly tech and city vs. country tech stories to swap.

More of my photos from the event available here: https://www.flickr.com/photos/pleia2/sets/72157651981993941/

Spring Trip to Philadelphia and New Jersey

I didn’t think I’d be getting on a plane at all in March, but plans shifted and we scheduled a trip to Philadelphia and New Jersey that left my beloved San Francisco on Sunday March 29th and returned us home on Monday, April 6th.

Our mission: Deal with our east coast storage. Without getting into the boring and personal details, we had to shut down a storage unit that MJ has had for years and go through some other existing storage to clear out donatable goods and finally catalog what we have so we have a better idea what to bring back to California with us. This required movers, almost an entire day devoted to donations and several days of sorting and repacking. It’s not all done, but we made pretty major progress, and did close out that old unit, so I’m calling the trip a success.

Perhaps what kept me sane through it all was the fact that MJ has piles of really old hardware, which is a delight to share on social media. Geeks from all around got to gush over goodies like the 32-bit SPARC lunchboxes (and commiserate with me as I tried to close them).


Notoriously difficult to close, but it was done!

Now admittedly, I do have some stuff in storage too, including my SPARC Ultra 10 that I wrote about here, back in 2007. I wanted to bring it home on this trip, but I wasn’t willing to put it in checked baggage and the case is a bit too big to put in my carry-on. Perhaps next trip I’ll figure out some way to ship it.


SPARC Ultra 10

More gems were collected in my album from the trip: https://www.flickr.com/photos/pleia2/sets/72157651488307179/

We also got to visit friends and family and enjoy some of our favorite foods we can’t find here in California, including east coast sweet & sour chicken, hoagies and chicken cheese steaks.

Family visits began on Monday afternoon as we picked up the plastic storage totes we were using to replace boxes, many of which were hard to go through in their various states of squishedness and age. MJ had them delivered to his sister in Pennsylvania and they were immensely helpful when we did the move on Tuesday. We also got to visit with MJ’s father and mother, and on Saturday met up with his cousins in New Jersey to have my first family Seder for Passover! Previously I’d gone to ones at our synagogue, but this was the first time I’d done one in someone’s home, and it meant a lot to be invited and to participate. Plus, the Passover diet restrictions did nothing to stem the exceptional dessert spread, there was so much delicious food.

We were fortunate to be in town for the first Wednesday of the month, since that allowed us to attend the Philadelphia area Linux Users Group meeting in downtown Philadelphia. I got to see several of my Philadelphia friends at the meeting, and brought along a box of books from Pearson to give away (including several copies of mine), which went over very well with the crowd gathered to hear from Anthony Martin, Keith Perry, and Joe Rosato about ways to get started with Linux, and freed up space in my closet here at home. It was a great night.


Presentation at PLUG

Friend visits included a fantastic dinner with our friend Danita and a quick visit to see Mike and Jessica, who had just welcomed little David into the world, awww!


Staying in New Jersey meant we could find Passover-friendly meals!

Sunday wrapped up with a late night at storage, finalizing some of our sorting and packing up the extra suitcases we brought along. We managed to get a couple hours of sleep at the hotel before our flight home at 6AM on Monday morning.

In all, it was a productive trip, but exhausting and I spent this past week making up for sleep debt and the aches and pains. Still, it felt good to get the work done and visit with friends we’ve missed.

Puppet Camp San Francisco 2015

On Tuesday, March 24th I woke up early and walked down the street to a regional Puppet Camp, this edition held not only in my home city of San Francisco, but just a few blocks from home. The schedule for the event can be found up on the Eventbrite page.

The event kicked off with a keynote by Ryan Coleman of Puppet Labs, who gave an overview of how configuration management tools like Puppet have shaped our current systems administration landscape. Our work will only continue to grow in scale as we move forward, and based on results of the 2014 DevOps Report more companies will continue to move their infrastructures to the cloud, where automation is key to a well-functioning system. He went on to talk about the work that has been going into Puppet 4 RC and some tips for attendees on how they can learn more about Puppet beyond the event, including free resources like Learn Puppet (which also links to paid training resources) and the Puppet Labs Documentation site, for which they have a dedicated documentation team working to make great docs.

Next up was a great talk by Jason O’Rourke of Salesforce who talked about his infrastructure of tens of thousands of servers and how automation using Puppet has allowed his team to do less of the boring, repetitive tasks and more interesting things. His talk then focused in on “Puppet Adoption in a Mature Environment” where he quickly reviewed different types of deployments, from fresh new ones where it’s somewhat easy to deploy a new management framework to old ones where you may have a lot of technical debt, compliance and regulatory considerations and inability to take risks in a production environment. He walked through the strategies they used to accomplished to make changes in the most mature environments, including the creation of a DevOps team who were responsible for focusing on the “infrastructure is code” mindset, use of tools like Vagrant so identical test environments can be deployed by developers without input from IT, the development of best practices for managing the system (including code review, testing, and more). One of the interesting things they also did was give production access to their DevOps team so they could run limited read/test-only commands against Puppet. This new system was then slowly rolled out typically when hardware or datacenters were rolled out, or when audits or upgrades are being conducted. They also rolled out specific “roles” in their infrastructure separately, from the less risky internal-only services to partner and customer-facing. The rest of the talk was mostly about how they actually deploy into production on a set schedule and do a massive amount of testing for everything they roll out, nice to see!


Jason O’Rourke of Salesforce

Tray Torrance of NCC Group rounded out the morning talks by giving a talk on MCollective (Marionette Collective). He began the talk by covering some history of the orchestration space that MCollective seeks to cover, and how many of the competing solutions are ssh-based, including Ansible, which we’ve been using in the OpenStack infrastructure. It was certainly interesting to learn how it integrates with Puppet and is extendable with Ruby code.

After lunch I presented a talk on “Puppet in the Open Source OpenStack Infrastructure” where I walked through how and why we have an open source infrastructure, and steps for how other organizations and projects can adopt similar methods for managing their infrastructure code. This is similar to some other “open sourcing all our Puppet” talks I have given, but with this audience I definitely honed in on the DevOps-y value of making the code for infrastructure more broadly accessible, even if it’s just within an organization. Slides here.

The next couple of talks were by Nathan Valentine and David Lutterkort of Puppet Labs. Nathan did several live demos of Puppet Enterprise, mostly working through the dashboard to demonstrate how services can be associated with servers and each other for easy deployment. David’s presentation went into a bit of systems administration history in the world before ever-present configuration management and virtualization to discuss how containerization software like Docker has really changed the landscape for testing and deployments. He walked through usage of the Puppet module for Docker written by Gareth Rushgrove and his cooresponding proof of concept for a service deployment in Docker for ASP.NET, available here.

The final talk of the day was by Aaron Stone (nickname “sodabrew”) of BrightRoll on “Dashboard 2 and External Node Classification” where he walked through the improvements to the Puppet Dashboard with the release of version 2. I myself had been exposed to Puppet Dashboard when I first joined the OpenStack Infrastructure team a couple years ago and we were using it to share read-only data to our community so we’d have insight into when Puppet changes merged and whether they were successful. Unfortunately, a period of poor support for the dashboard caused us to go through several ideas for an alternative dashboard (documented in this bug) until we finally settled on using a simple front end for PuppetDB, PuppetBoard. We’re really happy with the capabilities for our team, since read-only access is what we were looking for, but it was great to hear from Aaron about work he’s resumed on the Dashboard, should I have a need in the future. Some of the improvements he covered included some maintenance fixes, including broader support for newer versions of Ruby and updating of the libraries (gems) it’s using, an improved REST API and some UI tweaks. He said that upgrading should be easy, but in an effort to focus on development he wouldn’t be packaging it for all the distros, though the files (ie debian/ for .deb packages) to make this a task for someone else are available if someone is able to do the work.

In all, this was a great little event and the low ticket price of $50 it was quite the cost-effective way to learn about a few new technologies in the Puppet ecosystem and meet fellow, local systems administrators and engineers.

A few more photos from the event are here: https://www.flickr.com/photos/pleia2/sets/72157649225111213

Simcoe’s March 2015 Checkup

Our little Siamese, Simcoe, has Chronic Renal Failure (CRF). She has been doing well for over 3 years now with subcutaneous fluid injections every other day to keep her hydrated and quarterly check-ins with the vet to make sure her key blood levels and weight are staying within safe parameters.

On March 14th she went in for her latest visit and round of blood work. As usual, she wasn’t thrilled about the visit and worked hard to stay in her carrier the whole time.

She came out long enough for the exam, and the doctor was healthy with her physical, though her weight had dropped a little again, going from 9.74lbs to 9.54lbs.

Both her BUN and CRE levels remained steady.

Unfortunately her Calcium levels continue to come back a bit high, so the vet wants her in for an ionized Calicum test. She has explained that it’s only the ionized Calcium that is a concern because it can build up in the kidneys and lead to more rapid deterioration, so we’d want to get her on something to reduce the risk if that was the case. We’ll probably be making an appointment once I return from my travels in mid April to get this test done.

In the meantime, she gets to stay at home and enjoy a good book.

…my good book.

The spaces between

It’s been over 2 months since I’ve done a “miscellaneous life stuff” blog post. Anyone reading this blog recently might think I only write about travel and events! Since that last post I have had other things pop up here and there, but I am definitely doing too many events. That should calm down a bit in the 2nd quarter of the year and almost disappear in the third, with the notable exception of a trip to Peru, part work and part pleasure.

Unfortunately it looks like stress I mentioned in that last post flipped the switch on my already increasing-in-frequency migraines. I’ve seen my neurologist twice this year and we’ve worked through several medications, finally finding one that seems to work. And at least a visit to my neurologist affords me some nice views.

So I have been working on stress reduction, part which is making sure I keep running. It doesn’t reduce stress immediately but a routine of exercise does help even me out in the long term. To help clear my head, I’ve also been refining my todo lists to make them more comprehensive. I’m also continuing to let projects go when I find they’re causing my stress levels to spike for little gain. This is probably the hardest thing to do, I care about everything I work on and I know some things will just drop on the ground if I don’t do them, but I really need to be more realistic about what I can actually get done and focus my energy accordingly.

And to clear the way in this post for happier things, I did struggle with the loss of Eric in January. My Ubuntu work here in San Francisco simply won’t be the same without him, and every time I start thinking about planning an event I am reminded that he won’t be around to help or attend. Shortly after learning of his passing, several of us met up at BerkeleyLUG to share memories. Then on March 18th a more organized event was put together to gather friends from his various spheres of influence to celebrate his life at one of his favorite local pizzerias. It was a great event, I met some really good people and saw several old friends. It also brought some closure for me that I’d been lacking in dealing with this on my own.

On to happier things! I actually spent 30 days in a row off a plane in March. Home time means I got to do lots of enjoyable home things, like actually spending time with my husband over some fantastic meals, as well as finally finishing watching Breaking Bad together. I also think I’ve managed to somewhat de-traumatize my cats, who haven’t been thrilled about all my travel. We’ve been able to take some time to do some “home things” – like get some painting estimates so we can get some repairs done around the condo. I also spent a day down in Mountain View so I could meet up with a local colleague who I hadn’t yet met to kick off a new project, and then have dinner with a friend who was in the area visiting. Plus, I got to see cool things like a rare storm colliding with a sunset one evening:

I’ve been writing some, in January my article 10 entry points to tech (for girls, women, and everyone) went live on opensource.com. In early March I was invited to publish an article on Tech Talk Live Blog on Five Ways to Get Involved with Ubuntu as an Educator based on experience working with teachers over the past several years. I’ve also continued work toward a new book in progress, which has been time-consuming but I’m hoping will be ready for more public discussion in the coming months. Mark G. Sobell’s A Practical Guide to Ubuntu Linux, 4th Edition also came out earlier this year, and while I didn’t write that, I did spend a nice chunk of time last summer doing review for it. I came away with a quote on the cover endorsing the great work Mark did with the book!

Work-wise, aside from travel and conferences I’ve talked about in previous posts, I was recently promoted to root and core for OpenStack Infrastructure. This has meant a tremendous amount to me, both the trust the team has placed in me and the increased ability for me to contribute to the infrastructure I’ve spent so much time with over these past couple of years. It also means I’ve been learning a lot and sorting through the tribal knowledge that should be formally documented. I was also able to participate as a Track Chair for selecting talks for the Related OSS Projects track at the OpenStack Summit in Vancouver in May, I did this for Atlanta last year but ended up not being able to attend due to being too sick (stupid gallbladder). And while on the topic of Vancouver, a panel proposed by the Women of OpenStack that I’m participating in has been accepted, Standing Tall in the Room, where we hope to give other women in our community some tips for success. My next work trip is coming up before Vancouver I’m heading off to South Carolina for Posscon where I’ll be presenting on Tools for Open Source Systems Administration, a tour of tools we use in order to make collaborating online with a distributed team of systems administrators from various companies possible (and even fun!).

In the tech goodies department, I recently purchased a Nexus 6. I was compelled to after I dropped my Galaxy S3 while sitting up on the roof deck. I was pretty disappointed by the demise of my S3, it was a solid phone and the stress of replacement wasn’t something I was thrilled to deal with immediately upon my return from Oman. I did a bunch of research before I settled on the Nexus 6 and spent my hard-earned cash on retail price for a phone for the first time in my life. It’s now been almost a month and I’m still not quite used to how BIG the Nexus 6 is, but it is quite a pleasure to use. I still haven’t quite worked out how to carry it on my runs; it’s too big for my pockets and the arm band solution isn’t working (too bulky, and other reasons), I might switch to a small backpack that can carry water too. It’s a great phone though, so much faster than my old one, which honestly did deserve to be replaced, even if not in the way I face-planted it on the concrete, sorry S3.


Size difference: Old S3 in new Nexus 6 case

I also found my old Chumby while searching through the scary cave that is our storage unit for the paint that was used for previous condo painting. They’ve resurrected the service for a small monthly fee, now I just need to find a place to plug it in near my desk…

I actually made it out of the house to be social a little too. My cousin Steven McCorry is the lead singer in a band called Exotype, which signed a record deal last year and has since been on several tours. This one brought him to San Francisco, so I finally made my way out to the famous DNA Lounge to see the show. It was a lot of fun, but as much as I can appreciate metal, I’m pleased with their recent trend toward rock, which I prefer. It was also great to visit with my cousin and his band mates.

This week it was MJ’s turn to be out of the country for work. While I had Puppet Camp to keep me busy on Tuesday, I did a poor job of scheduling social engagements and it’s been a pretty lonely time. It gave me space to do some organization and get work done, but I wasn’t as productive as I really wanted to be and I may have binge watched the latest slew of Mad Men episodes that landed on Netflix one night. Was nice to have snuggle time with the kitties though.

MJ comes home Sunday afternoon, at which time we have to swap out the contents of his suitcase and head back to the airport to catch a red eye flight to Philadelphia. We’re spending next week moving a storage unit, organizing our new storage situation and making as many social calls as possible. I’m really looking forward to visiting PLUG on Wednesday to meet up with a bunch of my old Philadelphia Linux friends. And while I’m not actively looking forward to the move, it’s something we’ve needed to do for some time now, so it’ll be nice for that to be behind us.