• Archives

  • Categories

  • Other profiles

  • wallaceadngromit.net

  • Partimus

  • Secular Humanism

  • Debian

  • Xubuntu

  • Ubuntu California

  • OpenStack

My Yakkety Yak has arrived!

I like toys, but I’m an adult who lives in a small condo, so I need to behave myself when it comes to bringing new friends into our home. I made an agreement with myself to try and limit my stuffed toy purchases to two per year, one for each Ubuntu release.

Even so, I now have quite the collection.

These toys serve the purpose of brightening up our events with some fun, and enjoy the search for a new animal to match Mark Shuttleworth’s latest animal announcement. Truth be told, my tahr is a goat that I found that kind of looks like a tahr. The same goes for my xerus. My pangolin ended up having to be a plastic toy, though awareness about the animal (and conservation effords) has grown since 2012 so I’d likely be able to find one now. The quetzal was the trickiest, I had to admit defeat bought an ornament instead, but I did find and buy some quetzal earrings during our honeymoon in Mexico.

I’ve had fun as well and learned more about animals, which I love anyway. For the salamander I bought a $55 Hellbender Salamander Adoption Kit from the World Wildlife fund, an organization my husband and I now donate to annually. Learning about pangolins led me to visit one in San Diego and become a made me aware of the Save Pangolins organization.

It is now time for a Yakkety Yak! After some indecisiveness, I went with an adorable NICI yak, which I found on Amazon and shipped from Shijiazhuang, China. He arrived today.

Here he is!

…though I did also enjoy the first photo I took, where trusty photobombed us.

Newton OpenStack Summit Days 3-5

On Monday and Tuesday I was pretty focused on the conference side of the OpenStack Summit, but with all the keynotes behind us, when Wednesday rolled around I found myself much more focused on the Design Summit side.

Our first session of the day was on Community Task Tracking, which we jokingly called the “task tracking bake-off.” As background, couple years ago the OpenStack Infrastructure team placed our bets on an in-project developed task tracker called StoryBoard. The hope had been that the intention to move off of Launchpad and onto this new platform would bring support from companies looking to help with development. Unfortunately this didn’t pan out. Development landed on the shoulders of a single poor, overworked soul. At this point we started looking at the Maniphest component of Phabricator. Simultaneously we ended up with a contributor putting together configuration management for Maniphest and had a team pop up to continue support of StoryBoard for a downstream that had begun using it. A few weeks ago I organized a bug day where the team got together to do a serious once through of outstanding bugs and provide feedback to the StoryBoard team about what we need to use it, we went from 571 active bugs down to 414.

This set the stage for our session. We could stand up a Maniphest server or place our bets with StoryBoard again. We had a lot to consider.

  Pros Cons
Storyboard Strong influence over direction, already running and being used in our infra, good API We need to invest in development ourselves, little support for design/UI folks (though we could run a standalone Pholio)
Maniphest Investment is made by a large exiting development team, feature rich with pluggable components like Pholio for design folks Little influence over direction (like with Gerrit), still have to stand up and migrate to, weak API

Both had a few things lacking that we’d need before we go full steam into use by all of OpenStack, so there seemed to be consensus that they were similar in terms of work and time needed to get to that point. Of all the considerations, the need to develop our own vs. depending on upstream is the one that weighed most heavily upon me. Will companies really step up and help with development once we move everyone into production? What happens if our excellent current StoryBoard developers are reassigned to other projects? Having an active upstream certainly is a benefit. The session didn’t end with a formal selection, but we will be discussing it more over the next couple weeks so we can move toward making a recommendation to the Technical Committee (TC). Read-only session etherpad here.

The next session I attended was in the QA track, for the DevStack Roadmap. The session centered around finally making DevStack use Neutron by default. It’s been some time since nova-networking was deprecated, so this switch was a long time in coming. In addition to the technical components of this switch, there documentation needs to be updated around the networking decisions. Since I’ve just recently done some deep dives into OpenStack networking, somehow I ended up volunteering to help with this bit! Read-only session etherpad here.

Before the very busy lunch I had coming up, there was one more morning session, on Landing Page for Contributors. The current pages we have on the wiki, like the Main page on the wiki itself and the How To Contribute wiki aren’t the most welcoming of pages, they’re more walls of text that a new contributor has to sift through. This session talked through a lot of the tooling that could be used to make a more inviting, approachable page, drawing from other projects who have forged this path in the past. Of course it is also important that the content is reviewed and maintainable from the project perspective too, so something that can be held in revision control is key. Read-only session etherpad here.

As lunch rolled around I rushed upstairs to assist with the Git and Gerrit – Lunch and Learn. The event began by expecting and separating out about 1/3 of the folks in the room who hadn’t completed the prerequisites. It was the job of myself and the other helpers to start working with these folks to get their accounts set up and git-review installed. This wasn’t a trivial task, in spite of my intimate knowledge of how our system works and years of using it, almost all the attendees used Windows or Mac. I use Linux full time and we don’t maintain good (or any) documentation for in our development workflow for OpenStack development for these other operating systems.

A lot of folks did make it through configuration, and it was nice to be reminded about how our community is growing and that our tools need to do as well. A patch was submitted several months back to add a video of how to set things up on Windows, but that’s inconsistent with the rest of our documentation and has not been accepted. It would be great to see some folks using these other operating systems help us get the written documentation into better shape. Beyond the prerequisites, session leaders Amy Marrich and Tamara Johnston walked folks through setting up their environment, submitting a patch to the sandbox repo, submitting a test bug, reviewing a change and more. The slide deck they used has been uploaded to Amy’s AustinSummit GitHub project. I also took a few minutes to explain the Zuul Status page and a bit about each of the pipelines that a change may go through on the way to being merged.

Git and Gerrit – Lunch and Learn

Directly after lunch I was in another infrastructure session, this time to talk about Launch-Node, Ansible and Puppet. Launching new, long-lived servers in our infrastructure is one of those tasks that has remained frustratingly hands on. This manual work has been a time sink and a lot of it can be automated, so we as a team consider this situation a bug. Our Launch-Node script has been developed to start tackling this and the session went through some of the things we need to be careful of, including handling of DNS and duplicate hostnames (what if we’re spinning up a replacement server?), when do we unmount and disassociate cinder volumes and trove databases with the old server and bring them up on the new? Lots of great discussion around all of this was had. Fixes were already coming in by the end of this session and we have a good path moving forward. Read-only session etherpad here.

The next infrastructure session focused on Wiki upgrades. We’ve been struggling with spam problems for a several months. We need to do an upgrade to get some of the latest anti-spam tooling, which also requires upgrading the operating system in order to get a newer version of PHP. The people-power we have for this is limited, as we all have a lot of other projects on our plates. The session began with outlining what we need to do to get this done, and wound down with the proposal to shut down the wiki in a year. The OpenStack project has great, collaborative tooling for publishing documentation and things, we also use etherpads a lot for notes and to do lists, is there really still an active need for a wiki? Thierry Carrez sent an email today that started work on socializing our options, whether to carry on with the wiki or not. As the discussions continue on list, I hope to help in finding tooling for teams that need it and the current tools don’t satisfy. While we do that over the next year, Paul Belanger has bravely stepped forward to lead up the ongoing maintenance of the wiki until the possible retirement. Read-only session etherpad here.

Thursday morning kicked off bright and early with a session on Proposal jobs. As some quick background, proposal jobs are run on a privileged server in the OpenStack infrastructure that has the credentials to publish to a few places, like translations files up to Zanata. With this in mind, and as general good policy, we like to keep jobs we’re running here down to a minimum, using non-privileged servers as much as possible to complete tasks. The session walked through several of the existing jobs and news ones that were being proposed to sort through how they could be done differently, and make sure we’re all on the same page as a team when it comes to approving new jobs on these servers. Read-only session etherpad here.

It was then on to a session to “Robustify” Ansible-Puppet. Several months back we switched over to a system of triggering Puppet runs with Ansible instead of using the Puppetmaster software. This process quickly became complicated, so much so that even I struggled to trace the whole path of how everything works. Thankfully Monty Taylor and Spencer Krum started off the session by whiteboarding how everything works together, or doesn’t, as the case may be. It was a huge help to see it sketched out so that the pain points could be identified, one of those rare times when it was super valuable to be together in a room as a team rather than trying to explain things over IRC. We learned that inventory creation for Ansible is one of our pain points, but the complexity of the whole system has made fixing problems tricky, you pull one thread and something else gets undone! We also discussed the status of logging, and how we can better prepare for edge cases where things Really Go Wrong and we can’t access to the server to see the logs to find out what happened. There’s also some Puppetboard debugging to do, as folks rely on the data from that and it hasn’t been entirely accurate in reporting failures lately. In all, a great session, read-only session etherpad here.

Monty and Spencer explain our Ansible-Puppet setup

Next up for Infrastructure was a fishbowl session about OpenID/SSO for Community Systems. The OpenStack Foundation invested in the development of OpenStackID when few other options that fit our need were mature in this space. Today we have the option of using ipsilon, which has a bigger development community and is already in use by another major open source project (Fedora). The session outlined the benefits of consuming an upstream tool instead, including their development model, security considerations and general resources that have been spent to roll our own solution. The session also outlined exactly what our needs are to move all of our authentication away from Launchpad hosted by Canonical. I think it was a good session with some healthy discussion about where we are with our tooling, read-only session etherpad here.

I spent my time after lunch with the translations/internationalization (i18n) folks in a 90 minute work session on Translation Processes and Tools (read-only session etherpad here). My role in this session, along with Steve Kowalik and Andreas Jaeger was to represent the infrastructure team and the tooling we could provide to help the i18n team get their work done. Of particular focus were the translations check site that we need to work toward bringing online and our plan to upgrade Zanata, and the underlying operating system it’s running on. We also discussed some of the other requirements of the team, like automated polling of Active Technical Contributor (ATC) status for translators and improved statistics on Stackalytics for translations. Andreas was also able to take time to show off the new translations procedure for reno-driven release notes, which allows for translations throughout the cycle as they’re committed to the repositories rather than a mad rush to complete them at the end. It was also nice to catch up with Alex Eng from the Zanata team and former i18n PTL Daisy (Ying Chun Guo) who I had such a great time with in Tokyo, I wish I’d had more time to grab a meal with them.

In our final Infrastructure-focused session of the day, we met to discuss operating system upgrades. With the release of the latest Ubuntu LTS (16.04) the week prior to the summit, we find ourselves in a world of three Ubuntu LTS releases in the mix. We decided to first carve out some time to get all of our 12.04 systems upgraded to 14.04. From there we’ll work to get our Puppet modules updated and services prepared for running on 16.04. Of particular interest to me is getting the Zanata server on 16.04 soon so we can upgrade the version of Zanata that it’s running and requires a newer version of Java than 14.04 provides. We also spent a little time splitting out the easier servers to upgrade from the more difficult ones, especially since some systems have very little data and don’t actually need an in place upgrade, we can simply redeploy workers. We will do a more thorough evaluation when we’re closer to upgrade time, which we’re scheduling for some time this month. Read-only session etherpad here.

Thursday evening meant it was time for our Infrastructure Team dinner! Over 20 self-proclaimed infrastructure team members piled into cars to make it across town to Freedmans to enjoy piles of BBQ. I had to pass on all things bready (including beer) but later in the evening we made our way inside to the bar where we found agave tequila that was not forbidden for me. The rest was history. Lots of fun and great chats with everyone, including a bunch of non-infra people who had been clued into our late night shenanigans and decided to join us.

Infra evening gathering, photo by Monty Taylor

Friday was our day for team work session gatherings. Infrastructure ended up in room 404 (which, in fact, was difficult to find). Jeremy Stanley (fungi) kicked the day off by outlining topics for Infra and QA that we may find valuable to work on together while we were in the room. I worked on a few things with folks for about an hour before switching tracks to join my translations friends again over in their work session.

Steve, Andreas and I made our way over to the i18n session to chat with them about the ability to translate more things (like DevStack documentation) and to give them an update from our upgrades session for an idea of when they could expect the Zanata upgrade. Perhaps the most exciting part of the morning was their request for us to finally shut down the OpenStack Transifex project. We switched to Zanata when Transifex went closed source, but our hosted account has lingered around for a year since we’ve used it “just in case” we needed something from it. With two OpenStack cycles on Zanata behind us, it was time to shut it down. We were all delighted when we saw the email: [Transifex] The organization OpenStack has been deleted by the user jaegerandi.

Cheerful crowd of i18n contributors!

After one more lunch at Cooper’s BBQ, I made it back to the Infrastructure room for more afternoon work, but I could feel the cloud of exhaustion hitting me by then. Most of what I managed was informally chatting with my fellow contributors and sketching out work to be done rather than actually getting much done. There’d be plenty of time for that once I returned home!

I concluded my time in Austin with a few colleagues with a visit to the Austin Toy Museum, some leisurely time at the Blue Cat Cafe (my first cat cafe!) and a quiet sushi dinner. With that, another great OpenStack Summit was behind me. My flight home left at 6AM Saturday morning.

Edit: Infrastructure PTL Jeremy Stanley has also written summaries of sessions here: Newton Summit Infra Sessions Recap

Newton OpenStack Summit Days 1-2

This past week I attended my sixth OpenStack Summit. This one took us to Austin, Texas. I was last in Austin in 2014 when I quickly stopped by to give a talk at the Texas LinuxFest, but I wasn’t able to stay long during that trip. This trip gave me a chance (well, several) to finally have some local BBQ!

I arrived Sunday afternoon and took the opportunity to meet up with Chris Aedo and Paul Belanger, who I’d be on the stage with on Monday morning. We were able to do our first meetup together and do a final once through of our slides to make sure they had all the updates we wanted and we were clear on where the transitions were. Gathering at the convention center also allowed to pick up our badges before the mad rush that would come the opening of the conference itself on Monday morning.

With Austin being the Live Music Capital of the World, we were greeted in the morning by live music from the band Soul Track Mind. I really enjoyed the vibe it brought to the morning, and we had a show to watch as we settled in and waited for the keynotes.

Jonathan Bryce and Lauren Sell of the OpenStack Foundation opened the conference and gave us a tour of numbers. The first OpenStack summit was held in Austin just under six years ago with 75 people and they were proud to announce that this summit had over 7,500. It’s been quite the ride that I’m proud to have been part of since the beginning of 2013. In Jonathan’s keynote we were able to get a glimpse into the real users of OpenStack, with highlights including the fact that 65% of respondents to the recent OpenStack User Survey are using OpenStack in production and that half of the Fortune 100 companies are using OpenStack in some capacity. It was also interesting to learn how important the standard APIs for interacting with clouds was for companies, a fact that I always hoped would shine through as this open source cloud was being adopted. The video from his keynote is here: Embracing Datacenter Diversity.

As the keynotes continued the ones that really stood out for me were by AT&T (video: AT&T’s Cloud Journey with OpenStack) and Volkswagen Group (Driving the Future of IT Infrastructure at Volkswagen Group.

The AT&T keynote was interesting from a technical perspective. It’s clear that the rise of mobile devices and the internet of things has put pressure on telecoms to grow much more quickly than they have in the past to handle this new mobile infrastructure. Their keynote shared that they expected this to grow an additional ten times by 2020. To meet this need, the networking aspects of technologies like OpenStack are important to their strategy as they move away from “black box” hardware from networking vendors and to more software-driven infrastructure that could grow more quickly to fit their needs. We learned that they’re currently using 10 OpenStack projects in their infrastructure, with plans to add 3 more in the near future, and learned about their in house AT&T Integrated Cloud (AIC) tooling for managing OpenStack. When the morning concluded, all their work was rewarded with a Super User award, they wrote about here.

The Volkswagen Group keynote was a lot of fun. As the world of electric and automated cars quickly approaches they have recognized the need to innovate more quickly and use technology to get there. They still seem to be in the early days of OpenStack deployments, but have committed a portion of one of their new data centers to just OpenStack. Ultimately they see a hybrid cloud future, leveraging both public and private hosting.

The keynote sessions concluded with the announcement of the 2017 OpenStack Summit locations: Boston and Sydney!

Directly after the keynote I had to meet Paul and Chris for our talk on OpenStack Infrastructure for Beginners (video, slides). We had a packed room. I lead off the presentation by covering an overview of our work and by giving a high level tour of the OpenStack project infrastructure. Chris picked up by speaking to how things worked from a developer perspective, tying that back into how and why we set things up the way we did. Paul rounded out the presentation by diving into more of the specifics around Zuul and Jenkins, including how our testing jobs are defined and run. I think the talk went well, we certainly had a lot of fun as we went into lunch chatting with folks about specific components that they were looking either to get involved with or replicate in their own continuous integration systems.

Chris Aedo presenting, photo by Donnie Ham (source)

After a delicious lunch at Cooper’s BBQ, I went over to a talk on “OpenStack Stable: What It Actually Means to Maintain Stable Branches” by Matt Riedemann, Matthew Treinish and Ihar Hrachyshka in the Upstream Development track of the conference. This was a new track for this summit, and it was great to see how well-attended the sessions ended up being. The goal of this talk was to inform members of the community what exactly is involved in management of stable releases, which has a lot more moving pieces than most people tend to expect. Video from the session up here. It was then over to “From Upstream Documentation To Downstream Product Knowledge Base” by Stefano Maffulli and Caleb Boylan of DreamHost. They’ve been taking OpenStack documentation and adjusting it for easier and more targeted for consumption by their customers. They talked about their toolchain that gets it from raw source from the OpenStack upstream into the proprietary knowledge base at DreamHost. It’ll be interesting to see how this scales long term through releases and documentations changes, video here.

My day concluded by participating in a series of Lightning Talks. My talk was first, during which I spent 5 minutes giving a tour of status.openstack.org. I was inspired to give this talk after realizing that even though the links are right there, most people are completely unaware of what things like Reviewday (“Reviews” link) are. It also gave me the opportunity to take a closer, current look at OpenStack Health prior to my presentation, I had intended to go to “OpenStack-Health Dashboard and Dealing with Data from the Gate” (video) but it conflicted with the talk we were giving in the morning. The lightning talks continued with talks by Paul Belanger on Grafyaml, James E. Blair on Gertty and Andreas Jaeger on the steps for adding a project to OpenStack. The lightning talks from there drifted away from Infrastructure and into more general upstream development. Video of all the lightning talks here.

Day two of the summit began with live music again! It was nice to see that it wasn’t a single day event. This time Mark Collier of the OpenStack Foundation kicked things off by talking about the explosion of growth in infrastructure needed to support the growing Internet of Things. Of particular interest was learning about how operators are particularly seeking seamless integration of virtual machines, containers and bare metal, and how OpenStack meets that need today as a sort of integration engine, video here.

The highlights of the morning for me included a presentation from tcp cloud in the Czech Republic. They’re developing a Smart City in the small Czech city of Písek. He did an overview of the devices they were using and presented a diagram demonstrating how all the data they collect from around the city gets piped into an OpenStack cloud that they run. He concluded his presentation by revealing that they’d turned the summit itself into a mini city by placing devices around the venue to track temperature and CO2 levels throughout the rooms, very cool. Video of the presentation here.

tcp cloud presentation

I also enjoyed seeing Dean Troyer on stage to talk about improving user experience (UX) with OpenStackClient (OSC). As someone who has put a lot of work into converting documented commands in my book in an effort to use OSC rather than the individual project clients I certainly appreciate his dedication to this project. The video from the talk is here. It was also great to hear from OVH, an ISP and cloud hosting provider who currently donates OpenStack instances to our infrastructure team for running CI testing.

Tuesday also marked the beginning of the Design Summit. This is when I split off from the user conference and then spend the rest of my time in development space. This time the Design Summit was held across the street from the convention center in the Hilton where I was staying. This area of the summit takes us away from presentation-style sessions and into discussions and work sessions. This first day focused on cross-project sessions.

This was the lightest day of the week for me, having a much stronger commitment to the infrastructure sessions happening later in the week. Still, went to several sessions, starting off with a session led by Doug Hellmann to talk about how to improve the situation around global requirements. The session actually seemed to be an attempt to define the issues around requirements and get more contributors to help with requirements project review and to chat about improvements to tests. We’d really like to see requirements changes have a lower chance of breaking things, so trying to find folks to sign up to do this test writing work is really important.

I had lunch with my book writing co-conspirator Matt Fischer to chat about some of the final touches we’re working on before it’s all turned in. Ended up with a meaty lunch again at Moonshine Grill just across the street from the convention center, after which I went into a “Stable Branch End of Life Policy” session led by Thierry Carrez and Matt Riedemann. The stable situation is a tough one. Many operators want stable releases with longer lifespans, but the commitment from companies to put engineers on it is extremely limited. This session explored the resources required to continue supporting releases for longer (infra, QA, etc) and there were musings around extending the support period for projects meeting certain requirements for up to 24 months (from 18). Ultimately by the end of the summit it does seem that 18 months continues to be the release lifespan of them all.

I then went over to the Textile building across from the conference center where my employer, HPE, had set up their headquarters. I had a great on-camera chat with Stephen Spector about how open source has evolved from hobbyist to corporate since I became involved in 2001. I then followed some of the marketing folks outside to shoot some snippits for video later.

The day of sessions continued with a “Brainstorm format for design summit split event” session that talked a lot about dates. As a starting point, Thierry Carrez wrote a couple blog posts about the proposal to split the design summit from the user summit:

With these insightful blog posts in mind, the discussion moved forward on the assumption that the events would be split and how to handle that timing-wise. When in the cycle would each event happen for maximum benefit for our entire community? In the first blog post he had a graphic that had a proposed timeline, which the discussions mostly stuck to, but dove deeper into discussing what is going on during each release cycle week and what the best time would be for developers to gather together to start planning the next release. While there was good discussion on the topic, it was clear that there continues to be apprehension around travel for some contributors. There are fears that they would struggle to attend multiple events funding-wise, especially when questions arose around whether mid-cycle events would still be needed. Change is tough, but I’m on board with the plan to split out these events. Even as I write this blog post, I notice the themes and feel for the different parts of our current summit are very different.

My session day concluded with a session about cross-projects specifications for work lead by Shamail Tahir and Carol Barrett from the Product Working Group. I didn’t know much about OpenStack user stories, so this session was informative for seeing how those should be used in specs. In general, planning work in a collaborative way, especially across different projects that have diverse communities is tricky. Having some standards in place for these specs so teams are on the same page and have the same expectations for format seems like a good idea.

Tuesday evening meant it was time for the StackCity Community Party. Instead of individual companies throwing big, expensive parties, a street was rented out and companies were able to sponsor the bars and eateries in order to throw their branded events in them. Given my dietary restrictions this week, I wasn’t able to partake in much of the food being offered, so I only spent about an hour there before joining a similarly restricted diet friend over at Iron Works BBQ. But not before I picked up a dinosaur with a succulent in it from Canonical.

I called it an early night after dinner, and I’m glad I did. Wednesday through Friday were some busy days! But those days are for another post.

More photos from the summit here: https://www.flickr.com/photos/pleia2/albums/72157667572682751


A few weeks ago I had the pleasure of flying to Singapore to participate in FOSSASIA 2016, which is billed as Asia’s Premier Open Technology Event. I was able to spend a little time prior to the event doing some touristing but Friday morning came quickly and I met up with a colleague to make our way to the conference. We took the Singapore MRT (Mass Rapid Transit, rails!) from the station near our hotel to the Science Centre Singapore where the conference was being held. I was really pleased with how fast, frequent, clean and easy to navigate the MRT is during rush hour. Though the trains did tend to fill up, we had very easy rides to and from the venue each day.

This was my second open source conference in a science museum, and I really like the association. As conference attendees we were free to visit the museum (photos here). It was quite an honor to be welcomed to the center by Lim Tit Meng, the museum’s Chief Executive, during the keynotes on Friday morning. That morning I also had the pleasure of meeting FOSSASIA founder Hong Phuc, who I had been exchanging emails with leading up to the event, it was very clear that she’s continued to be very hands on with the organization of the conference since its founding.

The theme of the conference this year centered around the Internet of Things, so the Friday morning keynotes drew from a diverse group of people and organizations. I was particularly impressed that they didn’t just call upon open source developers to give presentations. Keynotes came from folks working on hardware, design and fascinating programs that used IoT devices.

Highlights of the morning included a talk by Bunnie Huang who made electronic, lighted badges for Burning Man that changed their light patterns based on how they “mated” with other badges to change their blinkome (think genome). Talks continued with a really fun one from Bernard Leong of the Singapore Post who explained how they’ve been experimenting with drones for small package delivery, particularly to remote areas, using Pulau Ubin as an example in the demonstration run.

I was then really delighted to hear about UNESCO’s YouthMobile program from Davide Storti and ITO Misako. YouthMobile is encouraging children to shift from being mere users of mobile devices to actually developing applications for them. I find this project to be particularly important as I know I wouldn’t be the technologist I am today without being able to fiddle with my early computers. We need to grow that next generation of tinkerers, but increasingly kids tend to only have access to mobile rather than the big old desktops that I grew up on. I believe projects aimed at inspiring the tinkerer in children on these new devices will grow in importance as we move into the future It was also nice to hear that the project hasn’t just been creating all their own curriculum to accomplish their goals, they’ve been partnering with existing initiatives and programs. Kudos to them for doing it right.

Davide Storti and ITO Misako on YouthMobile

Cat Allman continued keynotes as she talked about the work Google has started to do in the Maker and Science space. Their work includes Google Summer of Code accepting more science-focused programs, support of Maker events and “road trips” with students to science museums. The final keynote came from Jan Nikolai Nelles who spoke on the The Other Nefertiti, where a team visited a German museum and created a not-strictly-authorized 3D rendering of a famous Nefertiti bust. It was a valuable thing unto itself, and interesting for raising awareness about how museums share data about artifacts, or don’t, as the case may be.

The conference continued as I went to a talk titled “Why are we (Still) wasting food? How technology can help” which sounds interesting, but the presenter didn’t seem to understand his audience or what the conference was about. The talk was pretty much a sales talk about the success of their product in saving food in restaurant and other industry kitchens. A noble effort, and it was fun to brainstorm how some of the components he talked about could be used in other open source projects. I visited their website during the talk and was perplexed to be unable to find a link to their source code. During the Q&A I specifically asked whether the software was actually open source. The presenter struggled to answer my question, he claimed that it was, but that he is not a developer so he wasn’t sure which parts or where I could find it. He gave me his business card so I could send him an email about it after the conference. My email follow-up received this response:

“We are not using any open source code. Everything is developed in house.”

How disappointing! I’m not sure how their talk ended up at a Free and Open Source Software conference, though their selection of a non-technical presenter who couldn’t answer a simple question that strikes at the core of what the conference is about does hint at their obliviousness. I certainly didn’t appreciate being tricked into attending a sales talk about a suite of proprietary software. Thankfully, the conference improved after this.

I attended a talk by U-Zyn Chua about how he reverse engineered an API in a taxi app for his Singapore Taxi data project. His talk was fascinating for two reasons. First he walked us through the work that had to be done to use an undocumented API. Second, the data about taxis that he collected was fascinating, high traffic areas, times of days when taxis were busy. Plus, between this talk and the Singapore Post talk I learned a lot about the geography and population centers of Singapore.

Official Group Photograph - FOSSASIA 2016
Official Group Photograph – FOSSASIA 2016 by Michael Cannon

The conference continued the next day and I made sure I made time to attend Sayan Chowdhury’s “Dive deep into Fedora Infra” talk. Fedora was an early project on my open source infra list and it’s always exciting to chat with their engineers and swap running infra in the open stories. Sayan’s talk gave an overview of several of the key services that they’ve developed and deployed, including projects like Fedora Infrastructure Message Bus (fedmsg) which was also deployed by the open source infra team for the Debian project. Unfortunately I had to quickly depart from that talk in order to make it over to my own just after.

I gave a talk on “Code Review for DevOps” which I had a lot of fun modifying for the 20 minute slot and for a devops rather than systems administration audience. I put a firmer emphasis on the development of tooling in our team and was able to tighten up the presentation a lot to deliver a whirlwind tour of how we do almost everything through a code review system and with testing. Slides from the presentation are here (PDF).

Photo of my presentation by Dong (Vincent) Ma source)

I mentioned that my talk was 20 minutes long, and that makes this a good time to pause and reflect on that format. Almost all the talks at this conference were 20 minute slots, which is about half the length I’m accustom to. I really like this length. If a talk is not interesting, at least it’s short. If it is interesting, 20 minutes does actually give enough time for a good presentation. The schedule also allowed for 10 minutes between sessions so that people could get to their next room. In reality, all this timing this could have used a bit more policing. Q&As and even talks themselves by speakers used to longer slots frequently overflowed beyond their 20 minute window and frequently made it difficult to complete seeing one talk and getting out to the next. For a volunteer-run event, they did do a good job overall of sticking to at least the schedule of when talks started in each room, so if I planned accordingly I rarely missed the beginning of a talk in an alternate track because the schedule had drifted.

Saturday afternoon I spent some time going to lightning talks, including one about “Continuous Integration and Continuous Deployment (CI/CD) for Open Source and Free Software Development” by my colleague Dong Ma. With only 5 minutes, he was quickly able to contrast some of the features of the FOSSology open source CI/CD workflow with that of the model the OpenStack community has developed.

Dong Ma on open source CI/CD

I was then off to Sundeep Anand’s presentation, “Using Python Client to talk with Zanata Server.” Last autumn we launched translate.openstack.org running on Zanata and have been using the Java client along with a series of scripts to handle manipulation of the translations in the OpenStack project. It was interesting to learn about his strides with the Python client, which is making its way up to feature parity with the Java one. Since OpenStack itself is written in Python, switching to this Python client may make sense for us at some point, as it would make it easier for developers on our project to contribute to it. During his talk he also gave a demonstration of Zanata itself as he walked through the use of the client.

These talks were all very practical for me and applicable to my work, but that doesn’t mean I didn’t go off and have fun too. Later that afternoon I attended a talk on “A trip to Pluto with OpenSpace” where the team developing OpenSpace took public images of the Pluto flyby and gave us a demonstration of how their software worked to provide such a fascinating, animated demonstration. I also got to learn about the New Palmyra project where people are getting together to create 3D models of famous monuments in Syria that have been or are at risk of being destroyed by ongoing military conflict in the region. I also enjoyed learning about the passion that everyone on that team is bringing to the project, and I have a lot of respect for and interest in their goals of preserving history.

On Sunday the first talk I attended was by François Cartegnie on the newest features of the popular, cross-platform VLC software project. As a user of multiple platforms (Linux and Android) it was nice to hear that with the 3.0 release they’re aiming to standardize on that release number, as the differing version numbers across platforms have been confusing. He also spent a great deal of time explaining the challenges they continually overcome to be the best player on the market, including not just by supporting encoding standards, but by also supporting when those encoding standards are poorly or improperly implemented. This can’t be an easy task. I was also interested to learn that the uPnP support has also been revamped and should be working better these days.

My colleague and tourism buddy for the week Matthew Treinish spoke next, on “QA in the Open.” Drawing from his experience as the QA project lead for OpenStack for several cycles, he talked about the plugin-driven model that OpenStack QA has adopted. This model has helped individual projects take ownership of their testing requirements and has helped scale the very small core QA team, which now spans over a thousand repositories and dozens of projects that make up OpenStack.

Matthew Treinish on QA in the Open

Sunday afternoon had a talk that was one of the conference highlights for me: “Reproducible Builds – fulfilling the original promise of free software” by Chris Lamb. I had an interest in the topic before joining the session, but it was one of those talks where I was really pulled in and became even more interested in the topic. The idea on the surface seems pretty simple, you want to be able to exactly replicate builds over time and space. But there are a number of challenges to this when it comes to actually doing it, which he outlined:

  • Timestamps
  • Timezones and locales
  • Different versions of libraries
  • Non-deterministic file ordering
  • Users, groups, umask, environment variables
  • Random behavior (eg. hash ordering)
  • Build paths

Chris Lamb on Reproducible Builds

As soon as he enumerated these things it was obvious that they all would be problems, and still surprising that it would be so difficult. From this talk I learned about the reproducible-builds.org project which seeks to document and discuss these issues and find solutions for all of them. Additionally, Chris himself is a participant in Debian and he was able to share statistics about how most Debian packages are now being created in a way that adheres to the reproducible model. Very cool stuff, I hope to learn more about it.

My afternoon continued by attending a talk about btrfs by Anand Jain. His focus was basics and then on to upcoming features in development. The talk may have convinced me to start using it in a basic way on one of my systems soon, as the support for the core components is actually quite stable these days. I then went to an Asciidoc talk, where presenter George Goh compiled his presentation from Asciidoc just before he began presenting, nicely done! He stressed the importance of documentation and making it easy to keep updated, with automated updates of references to things like figures that live inline in the text. He also explored the use of template systems in Asciidoc to easily export portions of your document to different projects and organizations while preserving the appropriate branding for each.

In what seemed much too soon, the conference conclusion came on Sunday evening. There were thanks and words from several of the organizers. Words from the audience and various attendees were also spoke, my favorite of which came from young (middle school by my US-rendering) students visiting from Saudi Arabia. Several had feared that the conference would be boring and too technical for the level they were at, but they expressed excitement about how much fun they had and how many presenters had succeeded in presenting topics that they could understand. It was thrilling to hear this from these students, I want the future architects of our future to start young, be exposed to free and open source software and to be excited by the possibilities.

More of my photos from the event here: https://www.flickr.com/photos/pleia2/albums/72157666299641355

Thanks to all the organizers and volunteers for putting this conference together. I had a wonderful time and hope to participate again in the future!

Color an Ubuntu Xenial Xerus

Last cycle I reached out to artist and creator of Full Circle Magazine Ronnie Tucker to see if he’d create a coloring page of a werewolf for some upcoming events. He came through and we had a lot of fun with it (blog post here).

With the LTS release coming up, I reached out to him again.

He quickly turned my request around, and now we have a xerus to color!

Xerus coloring page
Click the image or here to download the full size version for printing.

Huge thanks to Ronnie for coming through with this, it’s shared with a CC-SA license, so I encourage people to print and share them at their release events and beyond!

While we’re on the topic of the our African ground squirrel friend, thanks to Tom Macfarlane of the Canonical Design Team I was able to update the Animal SVGs section of the Official Artwork page on the Ubuntu wiki. For those of you who haven’t seen the mascot image, it’s a real treat.

Xerus official mascot

It’s a great accompaniment to your release party. Download the SVG version for printing from the wiki page or directly here.

Tourist in Singapore

Time flies, and my recent trip to Singapore to speak at FOSSASIA snuck up on me. I wasn’t able to make time to do research into local attractions and so I found myself there the day before the conference I was there to attend with only one thing on my agenda, the Night Safari. MJ told me about it years ago when he visited Singapore and how he thought I’d enjoy it, given my love for animals and zoos.

I flew Singapore Air, frequently ranked the best airline in the world, and for good reason. Even in coach, the service is top notch and the food is edible, sometimes even good. My itinerary took me through Seoul on the way out, which felt the long way of doing things but my layover was short and I had a contiguous flight number, so passengers were mostly just shuffled through security and loaded onto the next plane. I seem to have cashed in all my travel karma this trip and ended up with an entire center row to myself, which meant I could lie down and get some sleep during the flights even though I was in coach. Heavenly! I arrived in Singapore at the bright and early time of 2AM and caught a taxi to my hotel. Thankfully I was able to get some sleep there too so I was ready for my jet lag adjustment day on Wednesday.

In the morning I met up with a colleague who was also in town for the conference. With neither of us having plans, I dragged him along with me as we bought tickets for the Night Safari that evening, including transport from a tour company that included priority boarding inside the park once we arrived. And then on to a touristy hop on/hop off bus to give us an overview of the city.

On the tourist bus!

The first thing I’ll say about Singapore: It’s hot and humid. I’m not built for this kind of weather. As much as I enjoyed my adventures, it was a struggle each day to keep up with my “I went to school in Georgia, this is fine!” colleague and to stay hydrated.

Then there’s their love for greenery. As a city-state there is a prevalence of what they refer to as the “concrete jungle” but they also seem keen on striking a balance. Many buildings have green gardens, and even full trees, on various balconies and roofs of their tall buildings. Even throughout areas of the city you could find larger green spaces than I’m accustom to seeing, bigger trees that they’ve clearly made an effort to make sure could still thrive. It was nice to see in a city.

The tourist bus took us through the heart of downtown where we were staying, then down to Chinatown, where the where we saw the Sri Mariamman Temple (which is actually a Hindu temple). The financial and districts were next, and then we decided to leave the bus for a time as we got to the Gardens by the Bay. This was a huge complex. There were several outdoor gardens with various themes, which surround the main area that has a couple indoor complexes as well as the outdoor tree-like structures that loom large, I got some great pictures of them.

We decided to go into the Cloud Forest, seeing as we were in town to speak about our work on cloud software. I was worried it would be even hotter inside, but it was amusing to discover that it was actually cooler, quite the welcome break for me. The massive dome structure enclosed what I would compare to the rain forest dome inside the California Academy of Sciences building in San Francisco, but much bigger and with a strong focus on flora rather than fauna. You enter the building at ground level and take the elevator to the top to walk down several stories through exhibits showing plant life of all sorts. It made for some nice views of the whole complex, and outside too.

After the dome, it was back out in the heat. We walked through some of the outdoor gardens before hopping on the tourist bus again. We took it through the Indian neighborhood where we saw the Sri Veeramakaliamman Temple and Arab section which included getting to see the beautiful Masjid Sultan (mosque), near where we had dinner later in the week at an Indian place that advertised being Halal.

By the time the bus got back to the stop near our hotel it was time for me to take a break before the Night Safari. We were being picked up for the safari at 6PM, which took us on a van to meet our bus that took us up to the part of the island where the Night Safari was. The tour guide gave an interesting take on history and the social benefits of living in Singapore on our journey up. It did make me reflect upon the fact that while there was traffic, the congestion was nothing like I’d expect for a city of Singapore’s size. I hadn’t yet experienced the public transit, but as I’d learn later in my trip it was quite good for the southern parts of the island.

The Night Safari! First impression: Tourist trap. But it got better. Once you make your way past the crowds, shops and food places, and beyond the goofy welcome show that has various animals doing tricks things get better. The adventure begins on a tram through the park. With the tour we didn’t have to wait in line, which when combined with the bus ride there, made it worth the extra fee for paying for the tour. The tram takes you through various habitats from around the world where nocturnal animals dwell. Big cats, various types of deer, wolves and hippos were among the star attractions. I was delighted to finally get to see some tahrs, which the last Ubuntu LTS release were named after.

After the tram tour I was feeling pretty tired, heat and jet lag hitting me hard. But I decided to go on a couple of the walking trails anyway. It was worth it. The walking trails are by far the best part of the park! More animals and getting to take the time as much time as you want to see the various animals. Exhaustion started hitting me when we completed half the trails, but I got to see fishing cats, otters, bats, a sleeping pangolin (another Ubuntu LTS animal!) and my favorite of the night, the binturong, otherwise known as a bear cat. I didn’t take any pictures of the animals, because night safari. By the end of our walking I was pretty tired and just wanted to get back to my bed, we forewent the tour bus back to the hotel and just got a taxi.

Thursday evening the first conference events kicked off with a hot pot dinner, but prior to that we had more time for touristing. During our city tour the day before I saw the Mint – Museum of Toys. Casting away thoughts of Toy Story 2’s plot line of being sold to a Japanese toy museum, I was delighted to visit an actual toy museum. Sadly, their floor on Space and Sci-Fi toys was closed, but the rest of the museum mostly made up for it. The open parts of the museum had 5 floors of toy displays spanning about one hundred years. Most of the toys were cartoon-related, with Popeye, super heroes, various popular Anime and Disney characters all making a respectable showing. Some of the toys packed into displays had surprisingly high appraisals attached to them, and there were notes here and there about their rarity. I had a lot of fun!

After toys, we decided to find lunch. It turns out that a number of places aren’t open for lunch, so we wandered around for a bit until around noon when we found ourselves in the Raffles Hotel courtyard in front of a menu that looked lovely for lunch. It was outdoors, so no escaping the heat, but the shade made things a bit more tolerable. It didn’t take long for us to eye the list of Slings on the cocktail menu and learn via a Google search that we were sitting where Singapore Slings were invented. How cool! Hydration took a back seat, I had to have a Singapore Sling were they were invented.

After lunch we continued our walk to make our way to the newly opened National Gallery. I had actually read about this one incidentally before arriving in Singapore, as it just opened in November and the opening was briefly covered in a travel magazine I read. This new gallery is housed in the historical former Supreme Court and City Hall buildings, and they didn’t do anything to hide this. Particularly in the Supreme Court building, it was very obvious that it was a courthouse, with much of what look like original benches throughout and rooms that still looked like court rooms with big wooden chairs and (jury?) boxes. In all, they were amazing buildings. The contents within made it that much better, these were some of the most impressive galleries I’ve ever had the pleasure of walking through. Art spanned centuries and styles of southern Asian talent, as well as art from colonials. I do admit enjoying the older, more realistic art rather than the modern and abstract, but there was something for everyone there. I’ll definitely go again the next time I’m in Singapore.

The National Gallery visit concluded my tourist adventures. That evening we met up with fellow FOSSASIA speakers at a hot pot restaurant not far from our hotel. It was my first time having hot pot, collecting raw meats, vegetables and fish from a buffet and dumping it in various boiling pots with seasonings was an experience I’m glad I had, but the weather got me there too. Sitting over a boiling pot in the evening heat and humidity certainly took its toll on me. Later in the week I had the opposite culinary adventure when I ended up at Swensen’s, an ice cream chain that started in San Francisco. I’d never been to the one in San Francisco, but apparently they’ve been a big hit in south Asia. It was fascinating to be in a San Francisco-themed restaurant and order a Golden Gate Bridge sundae while sitting halfway around the world from my city by the bay. Maybe I should visit the one in San Francisco now.

More photos from my tour around Singapore here: https://www.flickr.com/photos/pleia2/albums/72157666098884052

Two days isn’t nearly enough in Singapore. Even though I don’t shop (and shopping is BIG there!) I only got a small taste of what the city had to offer.

Next stop was on to the conference at the Singapore Science Centre, which was quite the inspired venue selection for an open source conference, especially one that attracted a number of younger attendees, but that’s a story for another day.

Wine and dine in Napa Valley

In 2008 when I was visiting MJ in my first trip to San Francisco we had plans to go up to Napa Valley. Given the distance and crowds, the driver MJ hired for the day made an alternate suggestion: “How about Sonoma Valley instead?” That day was the beginning of us being Sonoma Valley fans. Tastings weren’t over-crowded, the wine was excellent, at the time traffic was tolerable even coming back to the city. We visited a winery with a wine cave, where we’d get engaged three years later. Last year we joined a wine club, sealing our fate to visit regularly.

We never did make it to Napa, until a couple weeks ago.

For MJ’s birthday last year I promised him a meal at the most coveted restaurant in California, The French Laundry. I worked with a concierge to complete the herculean effort to get reservations, and then rescheduled a couple of times to work around our shifting travel schedules. Finally they were firmed up for Sunday, March 13th. The timing worked out, with all our travel lately we hadn’t seen much of each other, so it was a nice excuse to get out of town and spend the weekend together. We drove up Friday night and checked into the Harvest Inn, catching a late dinner at the lovely restaurant there, Harvest Table.

Dinner at Harvest Table

Saturday morning we began our wine trail. We didn’t have a lot of time to plan this trip, so we depended upon the recommendations of my recent house guest, George Mulak (and remotely, his wife Vicki), who supplied us with a list of their favorites. Their recommendations were spot on. Our first stop was Heitz Cellar which was conveniently almost across the street from where we were staying. They have a relatively small tasting area, and sadly when we arrived the skies had opened up to give us piles of rain, so there was no enjoying the grounds. They did have a couple things I really liked though. The first was a bit surprise, I don’t typically care for Zinfandels, but we bought a bottle of theirs, it was very good. Two bottles of their port also came home with us. Next on our list was one of several Rutherford <Noun> wineries, and we ended up at the wrong one, in what was a lovely mistake. We found ourselves at Rutherford Hill. a famous winery known for their Merlots, and I love Merlot. They also had wine caves and did tours! On the rainy day that it ended up being, a wine cave tour was a fantastic shelter from the weather. Our bartender and tour guide was super friendly and inviting and there’s a reason they are world-renowned: their wines are wonderful. We even joined their club.

Drinking wine in the Rutherford Hills wine caves

For lunch we went to Rutherford Grill, which we quickly noticed looked a lot like one of our Silicon Valley favorites, Los Altos Grill, and San Francisco haunt Hillstone. Turns out they’re all related. The familiarity was a welcome surprise, and an enjoyable lunch.

Wine adventures continued in the same parking lot as the grill when we made our way across to Beaulieu Vineyard (BV). I think planning ahead would have served us better here, we just did the basic tasting which was pretty run of the mill. A day with better weather and a planned historic wine tour would have been a better experience, maybe next time. From there we made our final stop of the day back near our hotel at Franciscan Estate Winery. We had a lovely time chatting with the Philadelphia-native pouring our wines and did a couple flights covering their range of types and qualities. A fine way to round out our afternoon. We picked up some snacks and water (time to hydrate!) at the lovely Dean & DeLuca shop (purveyors of fine food) and went back to the hotel to spend some time relaxing before dinner.

Final tasting of the day at Franciscan

In preparation for our exciting French Laundry reservation the following day, we booked late (9:45PM) dinner reservations at a related restaurant, Bouchon. Another French restaurant by Thomas Keller, the meal was delicious and the atmosphere was both fancy and casual, a lovely mix of how at home a really nice Napa Valley restaurant can make you feel. Highly recommended, and quite a bit easier to get reservations at than The French Laundry, though I still did need to plan a couple weeks ahead.

Appetizers at Bouchon

Sunday morning concluded our stay at the Harvest Inn. In spite of the rainy weekend, I did get to enjoy walking through their grounds a bit and appreciated the spacious room we had and the real wood fireplace. The location was great too, giving us a nice home base for the loop of wineries we visited. We’d stay here again. Check out was quick and then we were dressed up and on our way to the gem of our Napa adventure: Tasting menu lunch at The French Laundry!

In case I haven’t drilled this home enough, The French Laundry was named the Best Restaurant in the World multiple times. Even when it’s not at the top, looking at pretty much any top 10 lists for the past decade will see it listed as well. Going here was a really big, once in a lifetime, kind of deal.

The rainy weekend continued as we were seated downstairs and settled in with a glass of champagne to start our meal. A half bottle of red wine later joined us mid-meal. What struck me first about the meal there was the environment. French restaurants I’ve been to are either very modern or very stuffy, neither of which I’m a huge fan of. The French Laundry was a lovely mix of the two, much like Bouchon of the previous night, it seemed to reflect its home in Napa Valley. The restaurant was truly laundry themed in a very classy way, with a clothes pin as their logo and the lamps on the walls tastefully boasting clothes laundry symbols. The staff was professional, charming and witty. The food was spectacular, quickly making it into one of the top three meals I’ve ever had. The meal took about three hours, with small plates coming at a nice pace to keep us satisfied but also relaxed so we could enjoy the time there. I was definitely full at the end, especially after the stream of beautiful and delicious desserts that filled our table at the end. At the conclusion of the meal we were given a copy of the menu and gifted the wooden clothes pins that were at our table upon arrival. In all, it was an exceptional experience.

Meal at The French Laundry

With some time on our hands following our long lunch at The French Laundry we decided to add one more winery to our itinerary before driving home, Hagafen Cellars. Their wines are Kosher, even for Passover, which makes them great for us during that no-bread time and a star at the White House during major Jewish and Israeli-focused events. Best of all, their wines are wonderful. Having not grown up Jewish, I was not aware of the disappointment found with the standard Manischewitz wine until a couple years ago, so it was refreshing to learn we have other options during Passover! We were pretty close to joining their wine club, but in the end preferred making our own selections, and with a trunk full of wine we figured we’d had enough for now.

Final stop, Hagafen Cellars

With that, our fairy tale weekend together in Napa Valley came to a close. MJ flew out to Seattle that night for work. My trip to Singapore had me leaving the next morning.

More photos from our weekend in Napa Valley here: https://www.flickr.com/photos/pleia2/albums/72157665313725990

Six years in San Francisco

February 2016 marked six years of me living here in San Francisco. It’s hard to believe that much time has passed, but at the same time I feel so at home in my latest adopted city. I sometimes find myself struggling to remember what it was like to live in the suburbs, drive every day and not be able to just walk to the dentist, or take in the beautiful sights along the Embarcadero as I go for a run. I’ve grown accustom to the weather and seasons (or lack thereof), and barely think twice when making plans. Of course the weather will be beautiful!

I love you, California, I adore spending my time on The Dock of the Bay.

Our travel schedules this year have been a bit crazy though. I just returned from my second overseas conference of the year on Monday and MJ has been spending almost half his time time traveling with work. We’ve tried to plan things so that we’re not out of town at the same time, but haven’t always been effective. Plus, being out of town the same time is great for the cats and our need for a pet sitter, but it’s less great for getting time together. We ended up celebrating Valentine’s Day a day early, on February 13th, in order to work around these schedules and MJ’s plan to leave for a trip on Sunday.

It was a fabulous Valentine’s Day dinner though. We went to Jardinière over in Hayes Valley and both ordered the tasting menu, and I went with the wine pairing since I didn’t have a flight to catch the next day. Everything was exceptional, from the sea urchin to the beautifully prepared, marbled steak that melts in your mouth. I hope we can make it back at some point.

With MJ out of town I’ve had to fight the temptation to slip into workaholic mode. I definitely have a lot of work to do, especially as my for-real-this-time book deadline approaches. But I’ve grown appreciative of the need to take a break, and how it untangles the mind to be fresh again the next day and more effective at solving problems. On Presidents’ Day I treated myself to an afternoon at the zoo.

More photos from the zoo here: https://www.flickr.com/photos/pleia2/albums/72157662402671763

I’ve also gotten to make time to spend with friends here and there, recently making it out to the cinema with a friend to see the Oscar Nominated Animation Shorts. I grew to appreciate these shorts years ago after learning my beloved Wallace & Gromit films had been nominated and won in the past, but it had been some time since I’d gone to a theater to enjoy them.

While MJ has been in town, I’ve reflected on my six years here in the city and realized there were still things I’ve wanted to do in the area but haven’t had the opportunity to, so I’ve been slowly checking them off my list. Even small changes to accommodate new things have been worth it. One afternoon we took a slight detour from going to the Beach Chalet and instead went downstairs to the Park Chalet where we had never been before.

High Tide Hefeweizen at the Park Chalet

While on the topic of food, we also finally made it over to Zachary’s Chicago Pizza over in Oakland, near the Rockridge BART station. I’m definitely a New York pizza girl, but I hear so many good things about Zachary’s every time I moan about the state of California pizza. We went around 2:30 in the afternoon on a Saturday afternoon and were seated immediately. Eating there is a bit of an event, you order and wait a half hour for your giant wall of deep dish pizza to cook, I had the Spinach & Mushroom. The toppings and cheese are buried inside the pizza, with the sauce covering the top. It was really good, even if I could barely finish two pieces (leftovers!).

After Zachary’s I had planned to take BART up to downtown Berkeley to hit up a comic book store, since the one I used to go to here in San Francisco has closed due to increasing rent. I was delighted to learn that there was a comic book store within walking distance of where we already were. That’s how I was introduced to The Escapist in Berkeley, just over the Oakland/Berkeley border. I picked up most of the backlog of comics I was looking for, and then hit up Dark Carnival next door, a great Sci-Fi and Fantasy book store that I’d been to in the past. I’ll be returning to both stores in the near future.

And now it’s time to take an aforementioned break. Saturday off, here I come!

Xubuntu 16.04 ISO testing tips

As we get closer to the 16.04 LTS release, it’s becoming increasingly important for people to be testing the daily ISOs to catch any problems. This past week, we had the landing of GNOME Software to replace the Ubuntu Software Center and this will definitely need folks looking at it and reporting bugs (current ones tracked here: https://bugs.launchpad.net/ubuntu/+source/gnome-software)

In light of this, I thought I’d quickly share a few of my own tips and stumbling points. My focus is typically on Xubuntu testing, but things I talk about are applicable to Ubuntu too.

ISO testing on a rainy day

1. Downloading the ISO

Downloading an ISO every day, or even once a week can be tedious. Helpfully, the team provides the images via zsync which will only download the differences in the ISO between days, saving you a lot of time and bandwidth. Always use this option when you’re downloading ISOs, you can even use it the first time you download one, as it will notice that none exists.

The zsync URL is right alongside all the others when you choose “Link to the download information” in the ISO tracker:

You then use a terminal to cd into the directory where you want the ISO to be (or where it already is) and copy the zsync line into the terminal and hit enter. It will begin by examining the current ISO and then give you a progress bar for what it needs to download.

2. Putting the image on a USB stick

I have struggled with this for several releases. At first I was using UNetbootin (unetbootin), then usb-creator (usb-creator-gtk). Then I’d switch off between the two per release when one or the other wasn’t behaving properly. What a mess! How can we expect people to test if they can’t even get the ISO on a USB stick with simple instructions?

The other day flocculant, the Xubuntu QA Lead, clued me into using GNOME Disks to put an ISO on a USB stick for testing. You pop in the USB stick, launch gnome-disks (you’ll need to install the gnome-disk-utility package in Xubuntu), select your USB stick in the list on the left and choose the “Restore Disk Image…” option in the top right to select the ISO image you want to use:

I thought about doing a quick screencast of it, but Paul W. Frields over at Fedora Magazine beat me to it by more than a year: How to make a Live USB stick using GNOME Disks

This has worked beautifully with both the Xubuntu and Ubuntu ISOs.

3. Reporting bugs

The ISO tracker, where you report testing results, is easy enough to log into, but a fair number of people quit the testing process when it gets to actually reporting bugs. How do I report bugs? What package do I report them against? What if I do it wrong?

I’ve been doing ISO testing for several years, and have even run multiple events with a focus on ISO testing, and STILL struggle with this.

How did I get over it?

First, I know it’s a really long page, but this will get you familiar with the basics of reporting a bug using the ubuntu-bug tool: Ubuntu ReportingBugs

Often times being familiar with the basic tooling isn’t enough. It’s pretty common to run into a bug that’s manifesting in the desktop environment rather than in a specific application. A wallpaper is gone, a theme looks wrong, you’re struggling to log in. Where do those get submitted? And Is this bad enough for me to classify it as “Critical” in the ISO Tracker? This is when I ask. For Xubuntu I ask in #xubuntu-devel and for Ubuntu I ask in #ubuntu-quality. Note: people don’t hover over their keyboards on IRC, explain what you’re doing, ask your question and be patient.

This isn’t just for bugs, we want to see more people testing and it’s great when new testers come into our IRC channels to share their experiences and where they’re getting stuck. You’re part of our community :)

Simcoe thinks USB sticks are cat toys


I hope you’ll join us.

OpenStack infra-cloud sprint

Last week at the HPE offices in Fort Collins, members of the OpenStack Infrastructure team focused on getting an infra-cloud into production met from Monday through Thursday.

The infra-cloud is an important project for our team, so important that it has a Mission!

The infra-cloud’s mission is to turn donated raw hardware resources into expanded capacity for the OpenStack infrastructure nodepool.

This means that in addition companies who Contribute Cloud Test Resources in the form of OpenStack instances, we’ll be running our own OpenStack-driven cloud that will provide additional instances to our pool of servers we run tests on. We’re using the OpenStack Puppet Modules (since the rest of our infra uses Puppet) and bifrost, which is a series of Ansible playbooks that use Ironic to automate the task of deploying a base image onto a set of known hardware.

Our target for infra-cloud was a few racks of HPE hardware provided to the team by HPE that resides in a couple HPE data centers. When the idea for a sprint came together, we thought it might be nice to have the sprint itself hosted at an HPE site where we could meet some of the folks who handle servers. That’s how we ended up in Fort Collins, at an HPE office that had hosted several mid-cycle and sprint events for OpenStack in the past.

Our event kicked off with an overview by Colleen Murphy of work that’s been done to date. The infra-cloud team that Colleen is part of has been architecting and deploying the infra-cloud over the past several months with an eye toward formalizing the process and landing it in our git repositories. Part of the aim of this sprint was to get everyone on the broader OpenStack Infrastructure team up to speed with how everything works so that the infra cores could intelligently review and provide feedback on the patches being deployed. Colleen’s slides (available here) also gave us an overview of the baremetal workflow with bifrost, the characteristics of the controller and compute nodes, networking (and differences found between the US East and US West regions) and her strategy for deploying locally for a development environment (GitHub repo here). She also spent time getting us up to speed with the HPE iLO management interfaces that we’ll have to use if we’re having trouble with provisioning.

This introduction took up our morning. After lunch it was time to talk about our plan for the rest of our time together. We discussed the version of OpenStack we wanted to focus on and broadly how and if we planned to do upgrades, along with goals of this project. Of great importance was also that we built something that could be redeployed if we changed something, we don’t want this infrastructure to bit rot and cause a major hassle if we need to rebuild the cloud for some reason. We then went through the architecture section of the infra-cloud documentation to confirm that the assumptions there continued to be accurate, and made notes accordingly on our etherpad when they were not.

Our discussion then shifted into broad goals for our week, out came the whiteboard! It was decided that we’d focus on getting all the patches landed to support US West so that by the end of the sprint we’d have at least one working cloud. It was during this discussion that we learned how valuable hosting our sprint at an HPE facility was. An attendee at our sprint, Phil Jensen, works in the Fort Collins data center and updated us on the plans for moving systems out of US West. The timeline that he was aware of was considerably closer than we’d been planning on. A call was scheduled for Thursday to sort out those details, and we’re thankful we did since it turns out we had to effectively be ready to shut down the systems by the end of our sprint.

Goals continued for various sub-tasks in what coalesced in the main goal of the sprint: Get a region added to Nodepool so I could run a test on it.

Tuesday morning we began tackling our tasks, and at 11:30 Phil came by to give us a tour of the local data center there in Fort Collins. Now, if we’re honest, there was no technical reason for this tour. All the systems engineers on our team have been in data centers before, most of us have even worked in them. But there’s a reason we got into this: we like computers. Even if we mostly interact with clouds these days, a tour through a data center is always a lot of fun for us. Plus it got us out of the conference room for a half hour, so it was a nice pause in our day. Huge thanks to Phil for showing us around.

The data center also had one of the server types we’re using in infra-cloud, the HP SL390. While we didn’t get to see the exact servers we’re using, it was fun to get to see the size and form factor of the servers in person.

Spencer Krum checks out a rack of HP SL390s

Tuesday was spent heads down, landing patches. People moved around the room as we huddled in groups, and there was some collaborative debugging on the projector as we learned more about the deployment, learned a whole lot more about OpenStack itself and worked through some unfortunate issues with Puppet and Ansible.

Not so much glamour, sprints are mostly spent working on our laptops

Wednesday was the big day for us. The morning was spent landing more patches and in the afternoon we added our cloud to the list of clouds in Nodepool. We then eagerly hovered over the Jenkins dashboard and waited for a job to need a trusty node to run a test…

Slave ubuntu-trusty-infracloud-west-8281553 Building swift-coverage-bindep #2

The test ran! And completed successfully! Colleen grabbed a couple screenshots.

We watch on Clark Boylan’s laptop as the test runs

Alas, it was not all roses. Our cloud struggled to obey the deletion command and the test itself ran considerably slower than we would have expected. We spent some quality time looking at disk configurations and settings together to see if we could track down the issue and do some tuning. We still have more work to do here to get everything running well on this hardware once it has moved to the new facility.

Thursday we spent some time getting US East patches to land before the data center moves. We also had a call mid-day to firm up the timing of the move. Our timing for the sprint ended up working out well for the move schedule, we were able to complete a considerable amount of work at the sprint before the machines had to be shut down. The call was also valuable in getting to chat with some of the key parties involved and learn what we needed to hand off to them with regard to our requirements for the new home the servers will have, in an HPE POD (cool!) in Houston. This allowed us to kick off a Network Requirements for Infracloud Relocation Deployment thread and Cody A.W. Somerville captured notes from the rest of the conversation here.

The day concluded with a chat about how the sprint went. The feedback was pretty positive, we all got a lot of work done, Spencer summarized our feedback on list here.

Personally, I liked that the HPE campus in Fort Collins has wild rabbits. Also, it snowed a little and I like snow.

I could have done without the geese.

It was also enjoyable to visit downtown Fort Collins in the evenings and meet up with some of the OpenStack locals. Plus, at Coopersmith’s I got a beer with a hop pillow on top. I love hops.

More photos from the week here: https://www.flickr.com/photos/pleia2/sets/72157662730010623/

David F. Flanders also Tweeted some photos: https://twitter.com/dfflanders/status/702603441508487169