• Archives

  • Categories:

  • Other profiles

POSSCON 2015

This past week I had the pleasure of attending POSSCON in the beautiful capital city of South Carolina, Columbia. The great event kicked off with a social at Hickory Tavern, which I arranged to be at by tolerating a tight connection in Charlotte. It all worked out and in spite of generally being really shy at these kind of socials, I found some folks I knew and had a good time. Late in the evening several of us even had the opportunity to meet the Mayor of Columbia who had come down to the event and talk about our work and the importance of open source in the economy today. It’s really great to see that kind of support for open source in a city.

The next morning the conference actually kicked off. Organizer Todd Lewis opened the event and quickly handed things off to Lonnie Emard, the President of IT-oLogy. IT-oLogy is a non-profit that promotes initial and continued learning in technology through events targeting everyone from children in grade school to professionals who are seeking to extend their skill set, more on their About page. As a partner for POSSCON, they were a huge part of the event, even hosting the second day at their offices.

We then heard from aforementioned Columbia Mayor Steve Benjamin. A keynote from the city mayor was real treat, taking time out of what I’m sure is a busy schedule showed a clear commitment to building technology in Columbia. It was really inspiring to hear him talk about Columbia, with political support and work from IT-oLogy it sounds like an interesting place to be for building or growing a career in tech. There was then a welcome from Amy Love, the South Carolina Department of Commerce Innovation Director. Talk about local support! Go South Carolina!

The next keynote was from Andy Hunt, who was speaking on “A New Look at Openness” which began with a history of how we’ve progressed with development, from paying for licenses and compilers for proprietary development to the free and open source tool set and their respective licenses we work with today. He talked about how this all progresses into the Internet of Things, where we can now build physical objects and track everything from keys to pets. Today’s world for developers, he argued, is not about inventing but innovating, and he implored the audience to seek out this innovation by using the building blocks of open source as a foundation. In the idea space he proposed 5 steps for innovative thinking:

  1. Gather raw material
  2. Work it
  3. Forget the whole thing
  4. Eureka/My that’s peculiar
  5. Refine and develop
  6. profit!

Directly following the keynote I gave my talk on Tools for Open Source Systems Administration in the Operations/Back End track. It had the themes of many of my previous talks on how the OpenStack Infrastructure team does systems administration in an open source way, but I refocused this talk to be directly about the tools we use to accomplish this as a geographically distributed team across several different companies. The talk went well and I had a great audience, huge thanks to everyone who came out for it, it was a real pleasure to talk with folks throughout the rest of the conference who had questions about specific parts of how we collaborate. Slides from my presentation are here (pdf).

The next talk in the Operations/Back End track was Converged Infrastructure with Sanoid by Jim Salter. With SANOID, he was seeking to bring enterprise-level predictability, minimal downtime and rapid recover to small-to-medium-sized businesses. Using commodity components, from hardware through software, he’s built a system that virtualizes all services and runs on ZFS for Linux to take hourly (by default) snapshots of running systems. When something goes wrong, from a bad upgrade to a LAN infected with a virus, he has the ability to quickly roll users back to the latest snapshot. It also has a system for easily creating on and off-site backups and uses Nagios for monitoring, which is how I learned about aNag, a Nagios client for Android, I’ll have to check it out! I had the opportunity to spend more time with Jim as the conference went on, which included swinging by his booth for a SANOID demo. Slides from his presentation are here.

For lunch they served BBQ. I don’t really care for typical red BBQ sauce, so when I saw a yellow sauce option at the buffet I covered my chicken in that instead. I had discovered South Carolina Mustard BBQ sauce. Amazing stuff. Changed my life. I want more.

After lunch I went to see a talk by Isaac Christofferson on Assembling an Open Source Toolchain to Manage Public, Private and Hybrid Cloud Deployments. With a focus on automation, standardization and repeatability, he walked us through his usage of Packer, Vagrant and Ansible to interface with a variety of different clouds and VMs. I’m also apparently the last systems administrator alive who hadn’t heard of devopsbookmarks.com, but he shared the link and it’s a great site.

The rooms for the talks were spread around a very walkable area in downtown Columbia. I wasn’t sure how I’d feel about this and worried it would be a problem, but with speakers staying on schedule we were afforded a full 15 minutes between talks to switch tracks. The venue I spoke it was in a Hilton, and the next talk I went to was in a bar! It made for quite the enjoyable short walks outside between talks and a diversity in venues that was a lot of fun.

That next talk I went to was Open Source and the Internet of Things presented by Erica Stanley. I had the pleasure of being on a panel with Erica back in October during All Things Open (see here for a great panel recap) so it was really great running into her at this conference as well. Her talk was a deluge of information about the Internet of Things (IoT) and how we can all be makers for it! She went into detail about the technology and ideas behind all kinds of devices, and on slides 41 and 42 she gave a quick tour of hardware and software tools that can be used to build for the IoT. She also went through some of the philosophy, guidelines and challenges for IoT development. Slides from her talk are online here, the wealth of knowledge packed into that slidedeck are definitely worth spending some time with if you’re interested in the topic.

The last pre-keynote talk I went to was by Tarus Balog with a Guide to the Open Source Desktop. A self-confessed former Apple fanboy, he had quite the sense of humor about his past where “everything was white and had an apple on it” and his move to using only open source software. As someone who has been using Linux and friends for almost a decade and a half, I wasn’t at this talk to learn about the tools available, but instead see how a long time Mac user could actually make the transition. It’s also interesting to me as a member of the Ubuntu and Xubuntu projects to see how newcomers view entrance into the world of Linux and how they evaluate and select tools. He walked the audience through the process he used to select a distro and desktop environment and then all the applications: mail, calendar, office suite and more. Of particular interest he showed a preference for Banshee (reminded him of old iTunes), as well as digiKam for managing photos. Accounting-wise he is still tied to Quickbooks, but either runs it under wine or over VNC from a Mac.

The day wound down with a keynote from Jason Hibbets. He wrote The foundation for an open source city and is a Project Manager for opensource.com. His keynote was all about stories, and why it’s important to tell our open source stories. I’ve really been impressed with the development of opensource.com over the past year (disclaimer: I’ve written for them too), they’ve managed to find hundreds of inspirational and beneficial stories of open source adoption from around the world. In this talk he highlighted a few of these, including the work of my friend Charlie Reisinger at Penn Manor and Stu Keroff with students in the Asian Penguins computer club (check out a video from them here). How exciting! The evening wrapped up with an afterparty (I enjoyed a nice Palmetto Amber Ale) and a great speakers and sponsors dinner, huge thanks to the conference staff for putting on such a great event and making us feel so welcome.

The second day of the conference took place across the street from the South Carolina State House at the IT-oLogoy office. The day consisted of workshops, so the sessions were much longer and more involved. But the day also kicked off with a keynote by Bradley Kuhn, who gave a basic level talk on Free Software Licensing: Software Freedom Licensing: What You Must Know. He did a great job offering a balanced view of the licenses available and the importance of selecting one appropriate to your project and team from the beginning.

After the keynote I headed upstairs to learn about OpenNMS from Tarus Balog. I love monitoring, but as a systems administrator and not a network administrator, I’ve mostly been using service-based monitoring tooling and hadn’t really looking into OpenNMS. The workshop was an excellent tour of the basics of the project, including a short history and their current work. He walked us through the basic installation and setup, and some of the configuration changes needed for SNMP and XML-based changes made to various other parts of the infrastructure. He also talked about static and auto-discovery mechanisms for a network, how events and alarms work and details about setting up the notification system effectively. He wrapped up by showing off some interesting graphs and other visualizations that they’re working to bring into the system for individuals in your organization who prefer to see the data presented in less technical format.

The afternoon workshop I attended was put on by Jim Salter and went over Backing up Android using Open Source technologies. This workshop focused on backing up content and not the Android OS itself, but happily for me, that’s what I wanted to back up, as I run stock Android from Google otherwise (easy to install again from a generic source as needed). Now, Google will happily backup all your data, but what if you want to back it up locally and store it on your own system? By using rsync backup for Android, Jim demonstrated how to configure your phone to send backups to Linux, Windows and Mac using ssh+rsync. For Linux at least so far this is a fully open source solution, which I quite like and have started using it at home. The next component makes it automatic, which is where we get into a proprietary bit of software, Llama – Location Profiles. Based on various types of criteria (battery level, location, time, and lots more), Llama allows you to identify criteria of when it runs certain actions, like automatically running rsync to do backups. In all, it was a great and informative workshop and I’m happy to finally have a useful solution to pulling photos and things off my phone periodically without plugging it in and using MTP, which apparently I hate and so never I do it. Slides from Jim’s talk, which also include specific instructions and tools for Windows and Mac are online here.

The conference concluded with Todd Lewis sending more thanks all around. By this time in the day rain was coming down in buckets and there were no taxis to be seen, so I grabbed a ride from Aaron Crosman who I was happy to learn earlier was a local but had come from Philadelphia and we had great Philly tech and city vs. country tech stories to swap.

More of my photos from the event available here: https://www.flickr.com/photos/pleia2/sets/72157651981993941/

Spring Trip to Philadelphia and New Jersey

I didn’t think I’d be getting on a plane at all in March, but plans shifted and we scheduled a trip to Philadelphia and New Jersey that left my beloved San Francisco on Sunday March 29th and returned us home on Monday, April 6th.

Our mission: Deal with our east coast storage. Without getting into the boring and personal details, we had to shut down a storage unit that MJ has had for years and go through some other existing storage to clear out donatable goods and finally catalog what we have so we have a better idea what to bring back to California with us. This required movers, almost an entire day devoted to donations and several days of sorting and repacking. It’s not all done, but we made pretty major progress, and did close out that old unit, so I’m calling the trip a success.

Perhaps what kept me sane through it all was the fact that MJ has piles of really old hardware, which is a delight to share on social media. Geeks from all around got to gush over goodies like the 32-bit SPARC lunchboxes (and commiserate with me as I tried to close them).


Notoriously difficult to close, but it was done!

Now admittedly, I do have some stuff in storage too, including my SPARC Ultra 10 that I wrote about here, back in 2007. I wanted to bring it home on this trip, but I wasn’t willing to put it in checked baggage and the case is a bit too big to put in my carry-on. Perhaps next trip I’ll figure out some way to ship it.


SPARC Ultra 10

More gems were collected in my album from the trip: https://www.flickr.com/photos/pleia2/sets/72157651488307179/

We also got to visit friends and family and enjoy some of our favorite foods we can’t find here in California, including east coast sweet & sour chicken, hoagies and chicken cheese steaks.

Family visits began on Monday afternoon as we picked up the plastic storage totes we were using to replace boxes, many of which were hard to go through in their various states of squishedness and age. MJ had them delivered to his sister in Pennsylvania and they were immensely helpful when we did the move on Tuesday. We also got to visit with MJ’s father and mother, and on Saturday met up with his cousins in New Jersey to have my first family Seder for Passover! Previously I’d gone to ones at our synagogue, but this was the first time I’d done one in someone’s home, and it meant a lot to be invited and to participate. Plus, the Passover diet restrictions did nothing to stem the exceptional dessert spread, there was so much delicious food.

We were fortunate to be in town for the first Wednesday of the month, since that allowed us to attend the Philadelphia area Linux Users Group meeting in downtown Philadelphia. I got to see several of my Philadelphia friends at the meeting, and brought along a box of books from Pearson to give away (including several copies of mine), which went over very well with the crowd gathered to hear from Anthony Martin, Keith Perry, and Joe Rosato about ways to get started with Linux, and freed up space in my closet here at home. It was a great night.


Presentation at PLUG

Friend visits included a fantastic dinner with our friend Danita and a quick visit to see Mike and Jessica, who had just welcomed little David into the world, awww!


Staying in New Jersey meant we could find Passover-friendly meals!

Sunday wrapped up with a late night at storage, finalizing some of our sorting and packing up the extra suitcases we brought along. We managed to get a couple hours of sleep at the hotel before our flight home at 6AM on Monday morning.

In all, it was a productive trip, but exhausting and I spent this past week making up for sleep debt and the aches and pains. Still, it felt good to get the work done and visit with friends we’ve missed.

Puppet Camp San Francisco 2015

On Tuesday, March 24th I woke up early and walked down the street to a regional Puppet Camp, this edition held not only in my home city of San Francisco, but just a few blocks from home. The schedule for the event can be found up on the Eventbrite page.

The event kicked off with a keynote by Ryan Coleman of Puppet Labs, who gave an overview of how configuration management tools like Puppet have shaped our current systems administration landscape. Our work will only continue to grow in scale as we move forward, and based on results of the 2014 DevOps Report more companies will continue to move their infrastructures to the cloud, where automation is key to a well-functioning system. He went on to talk about the work that has been going into Puppet 4 RC and some tips for attendees on how they can learn more about Puppet beyond the event, including free resources like Learn Puppet (which also links to paid training resources) and the Puppet Labs Documentation site, for which they have a dedicated documentation team working to make great docs.

Next up was a great talk by Jason O’Rourke of Salesforce who talked about his infrastructure of tens of thousands of servers and how automation using Puppet has allowed his team to do less of the boring, repetitive tasks and more interesting things. His talk then focused in on “Puppet Adoption in a Mature Environment” where he quickly reviewed different types of deployments, from fresh new ones where it’s somewhat easy to deploy a new management framework to old ones where you may have a lot of technical debt, compliance and regulatory considerations and inability to take risks in a production environment. He walked through the strategies they used to accomplished to make changes in the most mature environments, including the creation of a DevOps team who were responsible for focusing on the “infrastructure is code” mindset, use of tools like Vagrant so identical test environments can be deployed by developers without input from IT, the development of best practices for managing the system (including code review, testing, and more). One of the interesting things they also did was give production access to their DevOps team so they could run limited read/test-only commands against Puppet. This new system was then slowly rolled out typically when hardware or datacenters were rolled out, or when audits or upgrades are being conducted. They also rolled out specific “roles” in their infrastructure separately, from the less risky internal-only services to partner and customer-facing. The rest of the talk was mostly about how they actually deploy into production on a set schedule and do a massive amount of testing for everything they roll out, nice to see!


Jason O’Rourke of Salesforce

Tray Torrance of NCC Group rounded out the morning talks by giving a talk on MCollective (Marionette Collective). He began the talk by covering some history of the orchestration space that MCollective seeks to cover, and how many of the competing solutions are ssh-based, including Ansible, which we’ve been using in the OpenStack infrastructure. It was certainly interesting to learn how it integrates with Puppet and is extendable with Ruby code.

After lunch I presented a talk on “Puppet in the Open Source OpenStack Infrastructure” where I walked through how and why we have an open source infrastructure, and steps for how other organizations and projects can adopt similar methods for managing their infrastructure code. This is similar to some other “open sourcing all our Puppet” talks I have given, but with this audience I definitely honed in on the DevOps-y value of making the code for infrastructure more broadly accessible, even if it’s just within an organization. Slides here.

The next couple of talks were by Nathan Valentine and David Lutterkort of Puppet Labs. Nathan did several live demos of Puppet Enterprise, mostly working through the dashboard to demonstrate how services can be associated with servers and each other for easy deployment. David’s presentation went into a bit of systems administration history in the world before ever-present configuration management and virtualization to discuss how containerization software like Docker has really changed the landscape for testing and deployments. He walked through usage of the Puppet module for Docker written by Gareth Rushgrove and his cooresponding proof of concept for a service deployment in Docker for ASP.NET, available here.

The final talk of the day was by Aaron Stone (nickname “sodabrew”) of BrightRoll on “Dashboard 2 and External Node Classification” where he walked through the improvements to the Puppet Dashboard with the release of version 2. I myself had been exposed to Puppet Dashboard when I first joined the OpenStack Infrastructure team a couple years ago and we were using it to share read-only data to our community so we’d have insight into when Puppet changes merged and whether they were successful. Unfortunately, a period of poor support for the dashboard caused us to go through several ideas for an alternative dashboard (documented in this bug) until we finally settled on using a simple front end for PuppetDB, PuppetBoard. We’re really happy with the capabilities for our team, since read-only access is what we were looking for, but it was great to hear from Aaron about work he’s resumed on the Dashboard, should I have a need in the future. Some of the improvements he covered included some maintenance fixes, including broader support for newer versions of Ruby and updating of the libraries (gems) it’s using, an improved REST API and some UI tweaks. He said that upgrading should be easy, but in an effort to focus on development he wouldn’t be packaging it for all the distros, though the files (ie debian/ for .deb packages) to make this a task for someone else are available if someone is able to do the work.

In all, this was a great little event and the low ticket price of $50 it was quite the cost-effective way to learn about a few new technologies in the Puppet ecosystem and meet fellow, local systems administrators and engineers.

A few more photos from the event are here: https://www.flickr.com/photos/pleia2/sets/72157649225111213

Simcoe’s March 2015 Checkup

Our little Siamese, Simcoe, has Chronic Renal Failure (CRF). She has been doing well for over 3 years now with subcutaneous fluid injections every other day to keep her hydrated and quarterly check-ins with the vet to make sure her key blood levels and weight are staying within safe parameters.

On March 14th she went in for her latest visit and round of blood work. As usual, she wasn’t thrilled about the visit and worked hard to stay in her carrier the whole time.

She came out long enough for the exam, and the doctor was healthy with her physical, though her weight had dropped a little again, going from 9.74lbs to 9.54lbs.

Both her BUN and CRE levels remained steady.

Unfortunately her Calcium levels continue to come back a bit high, so the vet wants her in for an ionized Calicum test. She has explained that it’s only the ionized Calcium that is a concern because it can build up in the kidneys and lead to more rapid deterioration, so we’d want to get her on something to reduce the risk if that was the case. We’ll probably be making an appointment once I return from my travels in mid April to get this test done.

In the meantime, she gets to stay at home and enjoy a good book.

…my good book.

The spaces between

It’s been over 2 months since I’ve done a “miscellaneous life stuff” blog post. Anyone reading this blog recently might think I only write about travel and events! Since that last post I have had other things pop up here and there, but I am definitely doing too many events. That should calm down a bit in the 2nd quarter of the year and almost disappear in the third, with the notable exception of a trip to Peru, part work and part pleasure.

Unfortunately it looks like stress I mentioned in that last post flipped the switch on my already increasing-in-frequency migraines. I’ve seen my neurologist twice this year and we’ve worked through several medications, finally finding one that seems to work. And at least a visit to my neurologist affords me some nice views.

So I have been working on stress reduction, part which is making sure I keep running. It doesn’t reduce stress immediately but a routine of exercise does help even me out in the long term. To help clear my head, I’ve also been refining my todo lists to make them more comprehensive. I’m also continuing to let projects go when I find they’re causing my stress levels to spike for little gain. This is probably the hardest thing to do, I care about everything I work on and I know some things will just drop on the ground if I don’t do them, but I really need to be more realistic about what I can actually get done and focus my energy accordingly.

And to clear the way in this post for happier things, I did struggle with the loss of Eric in January. My Ubuntu work here in San Francisco simply won’t be the same without him, and every time I start thinking about planning an event I am reminded that he won’t be around to help or attend. Shortly after learning of his passing, several of us met up at BerkeleyLUG to share memories. Then on March 18th a more organized event was put together to gather friends from his various spheres of influence to celebrate his life at one of his favorite local pizzerias. It was a great event, I met some really good people and saw several old friends. It also brought some closure for me that I’d been lacking in dealing with this on my own.

On to happier things! I actually spent 30 days in a row off a plane in March. Home time means I got to do lots of enjoyable home things, like actually spending time with my husband over some fantastic meals, as well as finally finishing watching Breaking Bad together. I also think I’ve managed to somewhat de-traumatize my cats, who haven’t been thrilled about all my travel. We’ve been able to take some time to do some “home things” – like get some painting estimates so we can get some repairs done around the condo. I also spent a day down in Mountain View so I could meet up with a local colleague who I hadn’t yet met to kick off a new project, and then have dinner with a friend who was in the area visiting. Plus, I got to see cool things like a rare storm colliding with a sunset one evening:

I’ve been writing some, in January my article 10 entry points to tech (for girls, women, and everyone) went live on opensource.com. In early March I was invited to publish an article on Tech Talk Live Blog on Five Ways to Get Involved with Ubuntu as an Educator based on experience working with teachers over the past several years. I’ve also continued work toward a new book in progress, which has been time-consuming but I’m hoping will be ready for more public discussion in the coming months. Mark G. Sobell’s A Practical Guide to Ubuntu Linux, 4th Edition also came out earlier this year, and while I didn’t write that, I did spend a nice chunk of time last summer doing review for it. I came away with a quote on the cover endorsing the great work Mark did with the book!

Work-wise, aside from travel and conferences I’ve talked about in previous posts, I was recently promoted to root and core for OpenStack Infrastructure. This has meant a tremendous amount to me, both the trust the team has placed in me and the increased ability for me to contribute to the infrastructure I’ve spent so much time with over these past couple of years. It also means I’ve been learning a lot and sorting through the tribal knowledge that should be formally documented. I was also able to participate as a Track Chair for selecting talks for the Related OSS Projects track at the OpenStack Summit in Vancouver in May, I did this for Atlanta last year but ended up not being able to attend due to being too sick (stupid gallbladder). And while on the topic of Vancouver, a panel proposed by the Women of OpenStack that I’m participating in has been accepted, Standing Tall in the Room, where we hope to give other women in our community some tips for success. My next work trip is coming up before Vancouver I’m heading off to South Carolina for Posscon where I’ll be presenting on Tools for Open Source Systems Administration, a tour of tools we use in order to make collaborating online with a distributed team of systems administrators from various companies possible (and even fun!).

In the tech goodies department, I recently purchased a Nexus 6. I was compelled to after I dropped my Galaxy S3 while sitting up on the roof deck. I was pretty disappointed by the demise of my S3, it was a solid phone and the stress of replacement wasn’t something I was thrilled to deal with immediately upon my return from Oman. I did a bunch of research before I settled on the Nexus 6 and spent my hard-earned cash on retail price for a phone for the first time in my life. It’s now been almost a month and I’m still not quite used to how BIG the Nexus 6 is, but it is quite a pleasure to use. I still haven’t quite worked out how to carry it on my runs; it’s too big for my pockets and the arm band solution isn’t working (too bulky, and other reasons), I might switch to a small backpack that can carry water too. It’s a great phone though, so much faster than my old one, which honestly did deserve to be replaced, even if not in the way I face-planted it on the concrete, sorry S3.


Size difference: Old S3 in new Nexus 6 case

I also found my old Chumby while searching through the scary cave that is our storage unit for the paint that was used for previous condo painting. They’ve resurrected the service for a small monthly fee, now I just need to find a place to plug it in near my desk…

I actually made it out of the house to be social a little too. My cousin Steven McCorry is the lead singer in a band called Exotype, which signed a record deal last year and has since been on several tours. This one brought him to San Francisco, so I finally made my way out to the famous DNA Lounge to see the show. It was a lot of fun, but as much as I can appreciate metal, I’m pleased with their recent trend toward rock, which I prefer. It was also great to visit with my cousin and his band mates.

This week it was MJ’s turn to be out of the country for work. While I had Puppet Camp to keep me busy on Tuesday, I did a poor job of scheduling social engagements and it’s been a pretty lonely time. It gave me space to do some organization and get work done, but I wasn’t as productive as I really wanted to be and I may have binge watched the latest slew of Mad Men episodes that landed on Netflix one night. Was nice to have snuggle time with the kitties though.

MJ comes home Sunday afternoon, at which time we have to swap out the contents of his suitcase and head back to the airport to catch a red eye flight to Philadelphia. We’re spending next week moving a storage unit, organizing our new storage situation and making as many social calls as possible. I’m really looking forward to visiting PLUG on Wednesday to meet up with a bunch of my old Philadelphia Linux friends. And while I’m not actively looking forward to the move, it’s something we’ve needed to do for some time now, so it’ll be nice for that to be behind us.

Elastic{ON} 2015

I’m finally home for a month, so I’ve taken advantage of some of this time to attend and present at some local events. The first of which was Elastic{ON}, the first user conference for Elasticsearch and related projects now under the Elastic project umbrella. The conference venue was Pier 27, a cruise terminal on the bay. It was a beautiful venue with views of the bay, and clever use for a terminal while there aren’t ships coming in.

The conference kicked off with a keynote where they welcomed attendees (of which there were over 1300 from 35 countries!) and dove into project history from the first release in 2010. A tour of old logos and websites built up to the big announcement, the “Elastic” rebranding, as the scope of their work now goes beyond search in the former Elasticsearch name. The opening keynotes continued with several leads from projects within the Elastic family, including updates from Logstash and Kibana.

At lunch I ended up sitting with 3 other women who were attending the conference on behalf of their companies (when gender ratios are skewed, this type of congregation tends to happen naturally). We all got to share details about how we were using Elasticsearch, so that was a lot of fun. One woman was doing data analysis against it for her networking-related work, another was using it to store metadata for videos and the third was actually speaking that afternoon on how they’re using it to supplement the traditional earthquake data with social media data about earthquakes at the USGS.

Track sessions began after lunch, and I spent my afternoon camped out in the Demo Theater. The first talk was by the Elastic Director of Developer Relations, Leslie Hawthorne. She talked about the international trio of developer evangelists that she works with, focusing on their work to support and encourage meetup groups worldwide, noting that 75 cities now have meetups with a total of over 17,000 individual members. She shared some tips from successful meetup groups, including offering a 2nd track during meetups for beginners, using an unconference format rather than set schedule and mixing things up sometimes with hack nights on Elastic projects. It was interesting to learn how they track community metrics (code/development stats, plus IRC and mailing list activity) and she wrapped up by noting the new site at https://www.elastic.co/community where they’re working to add more how-tos and on-ramping content, which their recent acquisition of Found, which has maintained a lot of that kind of material.


Leslie Hawthorn on “State of the Community”

The next session was “Elasticsearch Data Journey: Life of a Document in Elasticsearch” by Alexander Reelsen & Boaz Leskes. When documents enter Elasticsearch as json output from a service like Logstash, it can seem like a bit of a black box as far as what exactly happens to it in order for it to be added to Elasticsearch. This talk went through what happens. It’s first stored in Elasticsearch, where it’s stored node-wise is based on several bits of criteria analyzed upon bringing in, and the data is normalized and sorted. While the data is coming in, it’s stored in a buffer and also written to a transaction log until it’s actually committed to disk, at which time it’s still in the transaction log until it can be replicated across the Elasticsearch cluster. From there, they went into discussing data retrieval, cluster scaling and while stressing that replication is NOT backups, how to actually do backups of each node and how to restore from them. Finally, they talked about the data deletion process and how it queues data for deletion on each node in segments and noted that this is not a reversible option.

Continuing in “Life of” theme, I also attended “Life of an Event in Logstash” by Colin Surprenant. Perhaps my favorite talk of the day, Colin did an excellent job of explaining and defining all the terms he used in his talk. Contrary to popular belief, this isn’t just useful to folks new to the project, but as a systems administrator who maintains dozens of different types of applications over hundreds of servers, I am not necessarily familiar with what Logstash in particular calls everything terminology-wise, so having it made clear during the talk was great. His talk walked us through the 3 stages that events coming into Logstash go through: Input, Filter and Output, and the sized queues between each of them. The Input stage takes whatever data you’re feeding into Logstash and uses plugins to transform it into a Logstash event. The Filter stage actually modifies the data from the event so that the data is made uniform. The Output stage translates the uniform data into whatever output you’re sending it to, whether it’s STDOUT or sending it off to Elastisearch as json. Knowing the bits of this system is really valuable for debugging loss of documents, I look forward to having the video online to share with my colleagues. EDIT 3/20/2015: Detailed slides online here.


Colin Surprenant on “Life of an Event in Logstash”

I tended to a avoid many of the talks by Elasticsearch users talking about how they use it. While I’m sure there’s valuable insights to be gained by learning how others use it, we’re pretty much convinced about our use and things are going well. So use cases were fresh to me when the day 2 keynotes kicked off with a discussion with Don Duet, Co-head of Technology at Goldman Sachs. It was interesting to learn that nearly 1/3 of the employees at Goldman Sachs are in engineering or working directly with engineering in some kind of technical analysis capacity. They were also framed as very tech-conscious company and long time open source advocate. In exploring some of their work with Elasticsearch he used legal documents as an example: previously they were difficult to search and find, but using Elasticsearch an engineer was empowered to work with the legal department to make the details about contracts and more searchable and easier to find.

The next keynote was a surprising one, from Microsoft! As a traditional proprietary, closed-source company, they haven’t historically been known for their support of open source software, at least in public. This has changed in recent years as the world around has changed and they’ve found themselves needing to not only support open source software in their stacks but contributing to things like the Linux kernel as well. Speaker Pablo Castro had a good sense of humor about this all as he walked attendees through three major examples of Elasticsearch use at Microsoft. It was fascinating to learn that it’s used for content on MSN.com, which gets 18 billion hits per month. They’re using Elasticsearch on the Microsoft Dynamics CRM for social media data, and in this case their actually using Ubuntu as well. Finally, they’re using it for the search tool in their cloud offering, Azure. They’ve come a long way!


Pablo Castro of Microsoft

The final keynote was from NASA JPL. The room was clearly full of space fans, so this was a popular presentation. They talked about how they use Elasticsearch to hold data about user behavior from browsers on internal sites so they can improve them for employees. They also noted the terribly common practice of putting data (in this case, for the Mars rover) into Excel or Powerpoint and emailing it around as a mechanism for data sharing, and how they’ve managed to get this data into Elasticsearch instead, clearly improving the experience for everyone.

After the keynotes, it time to do my presentation! The title of my talk was “elastic-Recheck Yourself Before You Wreck Yourself: How Elasticsearch Helps OpenStack QA” and I can’t take credit for the title, my boring title was replaced by a suggestion from the talk selection staff. The talk was fun, I walked through our use of Elasticsearch to power our elastic-recheck (status page, docs) tooling in OpenStack. It’s been valuable not only for developer feedback (“your patch failed tests because of $problem, not your code”), but by giving the QA an Infrastructure teams a much better view into what the fleet of test VMs are up to in the aggregate so we can fix problems more efficiently. Slides from my talk are here (pdf).


All set up for elastic-Recheck Yourself Before You Wreck Yourself

Following my talk, ended up having lunch with the excellent Carol Willing. We got to geek out on all kinds of topics from Python to clouds as we enjoyed an outdoor lunch by the bay. Until it started drizzling.

The most valuable talk in the afternoon for me was “Resiliency in Elasticsearch and Lucene” with Boaz Leskes & Igor Motov. They began by talking about how with scale came the realization that more attention needed to be paid to recovering from various types of failures, and that they show up more often when you have more workers. The talk walked through various failures scenarios and how they’ve worked (and are working) on making improvements in these areas, including “pulling the plug” for a full shutdown, various hard disk failures, data corruption, and several types of cluster and HA failures (splitbrain and otherwise), out of memory resiliency and external pressures. This is another one I’m really looking forward to the video from.

The event wrapped up with a panel from GuideStar, kCura and E*Trade on how they’re using Elasticsearch and several “war stories” from their experiences working with the software itself, open source in general and Elastic the company.

In all, the conference was a great experience for me, and it was an impressive inaugural conference, though perhaps I should have expected that given the expertise and experience of the community team they have working there! They plan on doing a second one, and I recommend attendance to folks working with Elasticsearch.

More of my photos from the conference here: https://www.flickr.com/photos/pleia2/sets/72157650940379129/

Tourist in Muscat, Oman

I had the honor of participating in FOSSC Oman this February, which I wrote about here. Our gracious hosts were very accommodating to all of our needs, starting with arranging assistance at the airport and lodging at a nearby Holiday Inn.

The Holiday Inn was near the airport without much else around, so it was my first experience with a familiar property in a foreign land. It was familiar enough for me to be completely comfortable, but different enough to never let me forget that I was in a new, interesting place. In keeping with standards of the country, the hotel didn’t serve alcohol or pork, which was fine by me.

During my stay we had one afternoon and evening to visit the sights with some guides from the conference. Speakers and other guests convened at the hotel and boarded a bus which first took us to the Sultan Qaboos Grand Mosque. Visiting hours for non-Muslims were in the morning, so we couldn’t go inside, but we did get to visit the outside gardens and take some pictures in front of the beautiful building.

From there we went to a downtown area of Muscat and were able to browse through some shops that seemed aimed at tourists and enjoy the harbor for a bit. Browsing the shops allowed me to identify some of the standard pieces I may want to purchase later, like the style of traditional incense burner. The harbor was quite enjoyable, a nice breeze coming in to take the edge off the hot days, which topped out around 90F while we were there (and it was their winter!).

We were next taken to Al Alam Palace, where the Sultan entertains guests. This was another outside only tour, but the walk through the plaza up to the palace and around was well worth the trip. There were also lit up mountainside structures visible from the palace which looked really stunning in the evening light.

That evening we headed up to the Shangri-La resort area on what seemed like the outskirts of Muscat. It was a whole resort complex, where we got to visit a beach before meeting up with other conference folks for a buffet dinner and musical entertainment for the evening.

I really enjoyed my time in Oman. It was safe, beautiful and in spite of being hot, the air conditioning in all the buildings made up for the time we spent outdoors, and the mornings and evenings were nice and cool. There was some apprehension as it was my first trip to the middle east and as a woman traveling alone, but I had no problems and everyone I worked with throughout the conference and or stay was professional, welcoming and treated me well. I’d love the opportunity to go back some day.

More photos from my trip here: https://www.flickr.com/photos/pleia2/sets/72157650553216248/

FOSSC Oman 2015

This past week I had the honor of speaking at FOSSC Oman 2015 in Muscat, following an invitation last fall from Professor Hadj Bourdoucen and the organizing team. Prior to my trip I was able to meet up with 2013 speaker Cat Allman who gave me invaluable tips about visiting the country, but above all made me really excited to visit the middle east for the first time and meet the extraordinary people putting on the conference.


Some of the speakers and organizers meet on Tuesday, from left: Wolfgang F. Finke, Matthias Stürmer, Khalil Al Maawali, me and Hadj Bourdoucen

My first observation was that the conference staff really went out of their way to be welcoming to all the speakers, welcoming us at the hotel the day before the conference, making sure all our needs were met. My second was that the conference was that it was really well planned and funded. They did a wonderful job finding a diverse speaker list (both topic and gender-wise) from around the world. I was really happy to learn that the conference was also quite open and free to attend, so there were participants from other nearby companies, universities and colleges. I’ll also note that there were more women at this conference than I’ve ever seen at an open source conference, at least half the audience, perhaps slightly more.

The conference itself began on Wednesday morning with several introductions and welcome speeches from officials of Sultan Qaboos University (SQU), the Information Technology Authority (ITA) and Professor Hadj Bourdoucen who gave the opening FOSSC 2015 speech. These introductions were all in Arabic and we were all given headsets for live translations into English.

The first formal talk of the conference was Patrick Sinz on “FOSS as a motor for entrepreneurship and job creation.” In this talk he really spoke to the heart of why the trend has been leaning toward open source, with companies tired of being beholden to vendors for features, being surprised by changes in contracts, and the general freedom of not needing “permission” to alter the software that’s running your business, or your country. After a break, his talk was followed by one by Jan Wildeboer titled “Open is default.” He covered a lot in his talk, first talking about how 80% of most software stacks can easily be shared between companies without harming any competitive advantage, since everyone needs all the basics of hardware interaction, basic user interaction and more, thus making use of open source for this 80% an obvious choice. He also talked about open standards and how important it is to innovation that they exist. While on the topic of innovation he noted that instead of trying to make copies of proprietary offerings, open source is now leading innovation in many areas of technology, and has been for the past 5 years.

My talk came up right after Jan’s, and with a topic of “Building a Career in FOSS” it nicely worked into things that Patrick and Jan had just said before me. In this world of companies who need developers for features and where they’re paying good money for deployment of open source, there are a lot of jobs cropping up in the open source space. My talk gave a tour of some of the types of reasons one may contribute (aside from money, there’s passion for openness, recognition, and opportunity to work with contributors from around the world), types of ways to get involved (aside from programming, people are paid for deployments, documentation, support and more) and companies to aim for when looking to find a job working on open source (fully open source, open source core, open source division of a larger company). Slides from my talk are available here (pdf).

Directly following my talk, I participated in a panel with Patrick, Jan and Matthias (who I’d met the previous day) where we talked about some more general issues in the open source career space, including how language barriers can impact contributions, how the high profile open source security issues of 2014 have impacted the industry and some of the biggest mistakes developers make regarding software licenses.

The afternoon began with a talk by Hassan Al-Lawati on the “FOSS Initiative in Oman, Facts and Challenges” where he outlined the work they’ve been doing in their multi-year plan to promote the use and adoption of FOSS inside of Oman. Initiatives began with awareness campaigns to familiarize people with the idea of open source software, development of training material and programs, in addition to existing certificate programs in the industry, and the deployment of Open Source Labs where classes on and development of open source can be promoted. He talked about some of the further future plans including more advanced training. He wrapped up his talk by discussing some of the challenges, including continued fears about open source by established technologists and IT managers working with proprietary software and in general less historical demand for using open source solutions. Flavia Marzano spoke next on “The role and opportunities of FOSS in Public Administrations” where she drew upon her 15 years of experience working in the public sector in Italy to promote open source solutions. Her core points centered around the importance of the releasing of data by governments in open formats and the value of laws that make government organizations consider FOSS solutions, if not compel them. She also stressed that business leaders need to understand the value of using open source software, even if they themselves aren’t the ones who will get the read the source code, it’s important that someone in your organization can. Afternoon sessions wrapped up with a panel on open source in government, which talked about how cost is often not a motivator and that much of the work with governments is not a technical issue, but a political one.


FOSS in Government panel: David Hurley, Hassan Al-Lawati, Ali Al Shidhani and Flavia Marzano

The conference wrapped up with lunch around 2:30PM and then we all headed back to our hotels before an evening out, which I’ll talk more about in an upcoming post about my tourist fun in Muscat.

Thursday began a bit earlier than Wednesday, with the bus picking us up at the hotel at 7:45AM and first talks beginning at 8:30AM.

Matthias Stürmer kicked off the day with a talk on “Digital sustainability of open source communities” where he outlined characteristics of healthy open source communities. He first talked about the characteristics that defined digital sustainability, including transparency and lack of legal or policy restrictions. The characteristics of healthy open source communities included:

  • Good governance
  • Heterogeneous community (various motivations, organizations involved)
  • Nonprofit foundation (doing marketing)
  • Ecosystem of commercial service providers
  • Opportunity for users to get things done

It was a really valuable presentation, and his observations were similar to mine when it comes to healthy communities, particularly as they grow. His slides are pretty thorough with main points clearly defined and are up on slideshare here.

After his presentation, several of us speakers were whisked off to have a meeting with the Vice-chancellor of SQU to talk about some of the work that’s been done locally to promote open source education, adoption and training. Can’t say I was particularly useful at this session, lacking experience with formal public sector migration plans, but it was certainly interesting for me to participate in.

I then met up with Khalil for another adventure, over to Middle East College to give a short open source presentation to students in an introductory Linux class. The class met in one of the beautiful Open Source Labs that Hassan had mentioned in his talk, it was a real delight to go to one. It was also fascinating to see that the vast majority of the class was made up of women, with only a handful of men – quite the opposite from what I’m used to! My presentation quickly covered the basics of open source, the work I’ve done both as a paid and volunteer contributor, examples of some types of open source projects (different size, structure and volunteer to paid ratios) and common motivations for companies and individuals to get involved. The session concluded with a great Q&A session, followed by a bunch of pictures and chats with students. Slides from my talk are here (pdf).


Khalil and me at the OSL at MEC

My day wound down back at SQU by attending the paper sessions that concluded the conference and then lunch with my fellow speakers.

Now for some goodies!

There is a YouTube video of each day up, so you can skim through it along with the schedule to find specific talks:

There was also press at the conference, so you can see one release published on Zawya: FOSSC-Oman Kicks Off; Forum Focuses on FOSS Opportunities and Communities and an article by the Oman Tribune: Conference on open source software begins at SQU.

And more of my photos from the conference are here: https://www.flickr.com/photos/pleia2/sets/72157650553205488/

Wrap up of the San Francisco Ubuntu Global Jam at Gandi

This past Sunday I hosted an Ubuntu Global Jam at the Gandi office here in downtown San Francisco. Given the temporal proximity to a lot of travel, I’ve had to juggle a lot to make this happen, a fair amount of work goes into an event like this, from logistics of getting venue, food and drinks, and giveaways to the actual prep for the event and actually telling people about it. In this case we were working on Quality Assurance for Xubuntu (and a little Lubuntu on a PPC Mac).

It’s totally worth it though, so I present to you the full list of prep, should you wish to do a QA event in your region:

  • Secure venue: Completed in December (thanks AJ at Gandi!).
  • Secure refreshments funding: Completed in January via the Ubuntu donations funding.
  • Create LoCo Team Portal event and start sharing it everywhere (social media, friendly mailing lists for locals who may be interested). Do this for weeks!
  • Prepare goodies. I had leftover pens and stickers from a previous event. I then met up with Mark Sobell earlier in the week to have him sign copies of A Practical Guide to Ubuntu Linux, 4th Edition we received from the publisher (thank you Mark and Prentice Hall!).
  • Collect and stage all the stuff you’re bringing.
  • Print out test cases, since it can be tricky to juggle reading the test case while also navigating the actual test on their laptop.
  • Also print out signs for the doors at the venue.
  • Tour venue and have final chat with your host about what you need (plates, cups and utensils? power? wifi? projector?).
  • Send out last minute email to attendees as a reminder and in case of any last minute info.
  • Make sure dietary requirements of attendees are met. I did go with pizza for this event, but I made sure to go with a pizzeria that offered gluten free options and I prepared a gluten free salad (which people ate!).
  • Download and burn/copy the daily ISOs as soon as they come out on the day of the event, and put them on USB sticks or discs as needed: Xubuntu went on USB sticks, Lubuntu for PPC went on a CD-R (alternate) and DVD-R (desktop, currently oversized).
  • Bring along any extra laptops you have so folks who don’t bring one or have trouble doing testing on theirs can participate
  • Make penguin-shaped cookies (this one may be optional).

With all of this completed, I think the event went pretty smoothly. My Ubuntu California team mates James Ouyang and Christian Einfeldt met me at my condo nearby to help me carry over everything. AJ met us upon arrival and we were able to get quickly set up.

I had planned on doing a short presentation to give folks a tour of the ISO Tracker but the flow of attendees made it such that I could get the experienced attendees off and running pretty quick (some had used the tracker before) and by the time they were starting we had some newcomers joining us who I was able to guide one-on-one.

I did a lot of running around, but attendees were able to help out each other too, and it was a huge help to bring along some extra laptops. I was also surprised to see that another PPC Mac showed up at the event! I thought the one I brought would be the only one that would be used for Lubuntu. Later in the event we were joined by some folks who came over after the nearby BerkeleyLUG meeting wrapped up at 3PM, and caused us to push the event a full hour later than expected (thanks to AJ for putting up with us for another hour!).

Prior to the event, I had worried some about attendance, but throughout the event we had about 12 people total come and go, which was the perfect amount for me and a couple of other Ubuntu Members to manage so that attendees didn’t feel ignored as they worked through their tests. Post event, I’ve been able to provide some feedback to the Ubuntu Quality team about some snafus we encountered while doing testing. Hopefully these can be fixed next time around so other teams don’t run into the same issues we did.

Aside from some of the hiccups with the trackers, I received really positive feedback from attendees. Looking forward to doing this again in the future!

More photos from the event available here: https://www.flickr.com/photos/pleia2/sets/72157650663176996/

Afternoon in Brussels

My trip to Brussels for FOSDEM was a short one, I have a lot of work to do at home so it was impossible for me to make the case for staying more than three days. But since I got in early Friday morning, I did have Friday afternoon to do a bit of exploring.

First stop: get some mussels and frites!

For the rest of the afternoon I had planned on taking one of the tourist buses around town, but by the time I was ready to go it was 2PM and the last loop started at 2:30 that day, not giving me enough time to snag the last bus, and even if I had, where’s the fun in never getting off it? So I made my way toward Grand Place, where there were loads of shops, drinks and museums.

I decided to spend my afternoon at the Museum of the City of Brussels, which is dedicated to the history of the city and housed at Grand Place in the former King’s Mansion (Maison du Roi).

I’m glad I went, the museum had some beautiful pieces and I enjoyed learning about some of the history of the city. They were also running a special exhibit about the German occupation around World War I, which offered some interesting and sad insight into how the Belgians handled the occupation and the suffering endured by citizens during that time. Finally, I thoroughly enjoyed the browse through the amusing array of costumes made for the famous Manneken Pis.

The museum closed at 5PM and I made my way to visit the actual Manneken Pis fountain, located a few blocks south of the Grand Palace. It was starting to get quite chilly out and I was glad I had packed mittens. I snapped my photo of the fountain and then meandered my way back north until I found a little cafe where I got myself a nice cup of hot chocolate and warmed up while I waited for the Software Freedom Conservancy dinner at Drug Opera.

I also spent time scouring shop fronts for a Delirium Tremens stuffed toy elephant (as seen here). I saw one through a shop window the last time I was in Brussels in 2010, but it was late at night and the shop was closed. Alas, I never did find the elephant… until after dinner when I was walking back to my hotel once again late at night and the shop was closed! Argh! May we meet again some day, dear pink elephant.

In general the short length of the trip meant that I also didn’t get to enjoy many Belgian beers on my trip, quite the tragedy, but I did have to be alert for the actual conference I came to speak at and attend.

More photos from my tourist adventure here: https://www.flickr.com/photos/pleia2/sets/72157650562831526/