pleia2's blog Elizabeth Krumbach Joseph's public journal about open source, DevOps, beer, travel, pink gadgets and her life in the city where little cable cars climb halfway to the stars. 2017-11-16T19:37:28Z WordPress pleia2 <![CDATA[MesosCon EU 2017]]> 2017-11-16T19:37:28Z 2017-11-16T19:37:28Z I attended my first MesosCon was back in September in Los Angeles. In October I head the pleasure of participating in my second, this time the European version in Prague. As I’ve mentioned previously, this was a busy week for me. My participation in All Things Open kicked off the week, and I landed in Prague on Wednesday evening with just enough time to check into my room, say hello to some folks I knew who were in town for the Open Source Summit and then head off to a planned dinner with work folks at Restaurant Mlýnec.

The next morning arrived quickly as I was up and available for a 7:30AM breakfast with some of the MesosCon Europe keynote speakers. Part of the breakfast was spent chatting with the panelists who would join me on stage that morning to participate in a panel on “SMACK in the Enterprise” that I was moderating. At 8:30 it was time to go down to the keynote room and see that everything was on schedule for our 9AM start. The folks at the Linux Foundation do a great job coordinating these events, it was a pleasure working with them as a speaker and track lead throughout the event.

At 9AM Ben Hindman opened the conference with a talk on the current state of Apache Mesos, reflecting on past MesosCons and the past year of developments. Improvements in the platform have included things like nesting of containers, and the creation of the Container Storage Interface (CSI). Fault domains and the promise of multi-cloud also made an appearance in his keynote.

Directly following Ben’s talk was one that, as a pure open source enthusiast, made me really happy to attend. Rich Bowen of the Apache Software Foundation who came to talk to us about The Apache Way. Given my background and current role, I’m familiar with the history of the foundation and loosely keep tabs on their current work. Still, seeing a history presented with a message is always an enjoyable way to consume it, and Rich did a masterful job weaving the history in with where we are today, and explaining what “Apache” means in the Apache Mesos project. From bottom-up leadership to collaborative decision-making and they way they approach conflict-resolution, there’s a lot to admire. He also drove home the importance of transparency in a project, with no decisions being made privately or in ways that are difficult to document for future participants of the project. He also touched upon where the project was today, with over 100 projects within the Apache Software Foundation and the Apache License remaining strong in the industry.

The SMACK (Spark, Mesos, Akka, Cassandra, Kafka) in the Enterprise panel was next, and as moderator I was thrilled to be joined by representatives from Audi, Deutsche Telecom and ASML. From connected cars to how they’re making innovations on mobile networks, learning how companies are using Mesos and the entire stack to do fast data processing really brings all the work we do into focus. Innovation is happening across various industries as we all become more familiar with what we’ll need to succeed in a world with so much data coming at us, and I’m proud to be a part of it.

Thanks to the Apache Community for taking this photo (source)

The keynotes concluded with one from Netflix Engineering Director, Katharina Probst. Internally at Netflix they’ve built a massive infrastructure that is, at a high level, managed day to day by a relatively small team of Site Reliability Engineers. This has been accomplished through the tooling they built called Mantis, that allows not only for the essential autoscaling that a company like Netflix requires, with viewing numbers dropping considerably when each region has their work day, and picking up in the evening hours, but also real time testing and metrics from their platform. The key for them has been not only monitoring to detect that there is a problem somewhere, but finding out exactly where it is and reporting to the engineers what the problem may be, so a fix can be developed without a long session of troubleshooting first. As someone who used to work in operations, this was something I could really appreciate, I’ve spent many long evenings chasing down problems, evenings that could have been made much shorter with the addition of some automation to rule out the usual suspects quickly. The scale of their operations also never ceases to amaze me, and she was able to share some statistics around that as well.

This first day of the conference landed me as the track lead for the DevOps and Ops track. The day began with a talk from David vonThenen around external storage. In this talk he gave a cost/benefit evaluation of local versus external storage, and then, not keeping to just files, he ran the same evaluation when looking at storage for databases, both traditional like Postgres and MySQL and the newer distributed databases. Tying this in to Mesos, he touched upon CSI work and mentioned that REX-Ray is currently being used by DC/OS to handle the attachment of external volumes. These were interesting considerations in what has been a quickly growing part of our ecosystem as demands upon reliable storage solutions for containers quickly increase. His talk was followed by one from Ádám Sándor titled “Knee Deep in Microservices” and which had a Doom theme to demonstrate the new “demons” we have let loose as we’ve also taken all the benefits of migrating to microservices. He cited the key elements of DevOps, resilience, elasticity, resource abstraction, and tooling that helps us monitor containers as key improvements in the microservices platforms that are helping us tame the demons we have unleashed. It was also nice for me to hear these things, these massive distributed systems are complicated, we need to be continually improving our toolkit (weapons!) to effectively manage them.

After lunch Adam Bordelon and Alexander Rojas joined the track to give a popular talk titled “Mesos Security Exposed!” I won’t enumerate them all here, but by digging into how Mesos works, they were able to pull back the curtain and explain what needed securing in your cluster, from API endpoints to use of TLS and the handling of private data (secrets). Julien Stroheker then spoke on “Doing Real DevOps with DC/OS” where he gave a live demonstration of a continuous integration pipeline with Jenkins, Docker, Gradle and Vamp. The sessions for the day concluded with Zain Malik giving us a tour of mesos2iam, “IAM Credentials for Containers Running Inside a Mesos Cluster”.

But the day was not over! Appetizers and drinks at the attendee reception were directly followed by Town Halls, where attendees could casually gather in groups to talk about Mesos, Marathon, DC/OS or Kubernetes. My colleague Matt led the DC/OS session, with me taking notes and pitching in here and there. Our Town Hall began with introductions, with 15-20 people in attendence as the 90 minute evening session progressed we had a nice mix of people from a variety of unrelated industries. With the ice broken, it was easy to get people talking about pain points they had with DC/OS and we had Adam Bordelon in the room to give history and insight into specific features and concerns that people were bringing up. At the end of the session we had a nice list of shared struggles, but the attendees also were able to swap knowledge with each other. Empathy goes a long way, and in my own work I know how valuable it is to know that concerns you have are being shared and solved by others too.

Attendees file in for the DC/OS town hall Thursday evening

The Town Halls wrapped up at 8:30 and I headed straight to bed. Between the jet lag and the 13+ hour day, I was exhausted.

On Friday the keynotes also began at 9AM, but the keynote slot was shorter in order to squeeze more track sessions in during the rest of the day.

After an welcome and opening remarks from Jörg Schad, the first keynote came from Yrieix Garnier who gave a more Enterprise-focused take on “The Future of Apache Mesos and DC/OS” than Ben had explored in his keynote the day before. He was able to pull up statistics about the data-processing power of the platforms, sharing that 50% of DC/OS clusters were running some form of the SMACK or ELK stacks. The big news from his talk was unveiling of TensorFlow support in DC/OS. We then had Pierre Cheynier join us to share a talk that he had originally proposed during our CFP, but we upgraded to a keynote, “Operating 600+ Mesos Servers on 7 Datacenters @Criteo”. The ability to scale is a key feature of Mesos, so it was fascinating to learn about the scale of their work and just how much data they were storing (171PB on Hadoop!). He also shared a series of tips for other organizations looking to operate at this scale, including effectively automating everything (configuration management, scaling, CI system), defensive configuration (things will go wrong, be prepared), visibility to operations as to what is going on (metrics, alerts, tracing), and the importance of doing networking right, and addressing problems like QoS and “noisy neighbors” during design. He also covered some incidents which gave further insight into the sorts of things they were able to effectively prevent, or not.

In a slight shift from Thursday, I spent Friday as the track lead for the Users and Ops track. The first talk came from the oldest company in the lineup, the publisher Houghton Mifflin Harcourt. I was looking forward to this talk, and as Robert Allen got into the details of “How HMH Went from Months to Minutes for Infrastructure Delivery” I was not disappointed. In a common theme for the day, he brought up the slow, inconsistent, old technology team was not keeping up with either the industry or their own product lines, as the publishing industry is rapidly changing to serve a more tech savvy customer base. He walked us through the creation of their “Bedrock tech services” team, and their DevOps focused goals, including comprehensive developer involvement from idea to production, a continuous delivery approach that encouraged small, frequent updates, and a change in culture that made them shift from feeling like they must prevent failure, and instead acting as if failure will occur and planning accordingly. He then dove into the technologies used, beyond Apache Mesos, they’ve also been using Apache Aurora, Terraform for orchestration, Vault for secrets, and a Jenkins plus Artifactory CI/CD pipeline. He also stressed the importance of metrics and logging, all things close to my own interests as well!

Tim Nolet then joined us to give a talk on “Advanced Deployment Strategies and Workflows for Containerized Apps on DC/OS” where he too walked us down the painful memory lane of the massive, error-prone deployments of the 2000s, the rise of DevOps and today the productization of a lot of the DevOps tooling, including Vamp, which he works on the development of. As a product, Vamp seeks to simplify and tame the ecosystem of microservices you have running by simplifying the process of deployments. He then showed a demo of load being distributed across different versions of a deployed application, as well as different versions being served up to customers using different clients.

In our next user story, we heard from Jay Chin on “Optimising Mesos Utilization at Opentable”. He began his talk with a quick production history of the infrastructure at OpenTable, sharing that they made the move to microservices in 2013. During this time they were orchestrating their microservices via standard configuration management tooling, a process that turned out to not only difficult to maintain at scale, but was actively disliked by the engineers who had to work on it. In 2014 they switched to Mesos, and through resource abstraction and running a consolidated cluster, they were able to simplify application-level operations. Additionally, it allowed them to easily create centralized metrics and logging. It’s a story I’d heard before from other companies, and one I was thrilled to hear again, but where OpenTable really led the way here was by open sourcing some interesting tools they were using, including the following mentioned during his talk:

The day continued with Ilya Dmitrichenko on “Time Traveling in the Universe of Microservices and Orchestration” where he used his own career as a baseline for the changes he was seeing in the industry with regard to the rise of microservices. His dabbling with old Sun boxes and awareness of things like openvz is consistent with some of my own hobbiest and day job work early in my career. Indeed, containers have always been with us. He went on to track the rise of Docker simplifying the space and tooling coming together to make management of a microservices infrastructure feasible for smaller organizations. The culmination of this story was his current work at Weaveworks which, just like Vamp improves deployments, improves and simplifies your network policy.

Jorge Salamero of Sysdig joined us next with the amusingly titled talk, “WTF, My Container Just Spawned a Shell”. I was immediately fond of this talk because after a sea of Macs, Jorge uses Xubuntu. He began his presentation by talking about how we consume container images, and the rise in static scanning of these images for libraries with vulnerabilities and more, but that this only goes so far. There are things that these scans miss because they need constant updating, and which behavioral analysis while the container is running will catch. He introduced how Sysdig’s metrics tooling used with Falco can give you a comprehensive view inside the containers you’re running. Suddenly you have access to security tracking that can show you every command taking place inside of your containers, and from there you can train it to be aware of problem behavior. He also talked about SysDig Inspect, the open source project that they have built their Sysdig Secure product with.

Julien Stroheker then joined us again to talk about his DC/OS autoscaler. This is a talk I’d already seen twice, so I won’t go into it, but it is a cool project that he’s always looking for help on! The final talk of the day came to us from Fredrik Lindner of Tunstall Nordic AB, who shared “There and Back Again: How Tunstall Healthcare Built an IoT Platform for Health Monitoring Using Mesos Cluster on Azure.” Just like so many other industries, elder care is undergoing a transformation with the help of IoT technology. He shared details of their “Evity” platform they were developing on top of DC/OS to help manage the data coming in from these devices so they can effectively and quickly meet the needs of the families they work with. It was a good story to end with, as with much of the technology we saw showcased throughout this MesosCon, as consumers we just assume it will all work. My FitBit will reliably track my steps and share them with my friends, my car will show me traffic, and when something goes wrong it’s unexpected and unsettling. This is even more pronounced with you get into the space of technologies that help with things like healthcare, where reliability, accuracy and speed have even more urgency. We’re building the platform that people are using to make sure these all run well, and that’s pretty exciting.

When I finally stepped out of that track, all the booths had been taken down and most of the conference attendees had gone their own ways. I was able to finally leave the hotel with Matt and Jörg to explore some of the old city in what turned into a lovely night that thankfully didn’t keep me out too late. The conference was great, but predictably exhausting, especially coming on the heels of All Things Open on another continent.

Huge thanks to everyone involved in organizing, and to all the speakers in my tracks who made the event really interesting by sharing their stories, tools and expertise. This event was a little smaller than the one in Los Angeles, but it didn’t feel like it, and the quality of the event was top notch.

More of my photos from the event can be found here: and more photos, slides and videos are hosted by the Linuxd Foundation at

pleia2 <![CDATA[Tourist in Prague]]> 2017-11-03T18:40:44Z 2017-11-03T18:40:44Z I’m in the midst of a bunch of travel for work, with a trip to Prague directly following the fun I had in Raleigh at All Things Open. I took a flight out of Raleigh on Tuesday evening, making connections in Charlotte and London before finally landing in Prague on schedule on Wednesday evening. I was there for MesosCon EU, but I’ll write about that later, first I want to talk about what I got to see Friday evening and on the Saturday I spent there to do some tourist things.

The Open Source Summit was happening when I got into town, so I knew a whole lot of people who were there for that event. I was thrilled to meet up with Gina Feichtinger, who I’d long known online through LinuxChix, but never had the opportunity to meet. Unfortunately our schedules didn’t work out to spend more than a few minutes chatting before I had to run off to dinner on Wednesday night, but that was enough time to get a picture!

The conference concluded on Friday night and I was able to get my first real taste of the city as I went out with some colleagues for drinks at Lokal and then a nice dinner together over at U Sádlů. Prague is beautiful at night, and I enjoyed wandering around Old Town all evening.

Looking over at Church of Our Lady before Týn (not a castle!)

Saturday was my big tourist day. I met up with my colleagues Matt and Jörg to head up to Prague Castle. Prague Castle does not look like a castle, I’d call it a palace. There were tours but with just a couple hours of morning on our hands we decided to just explore the outside. We did make our way inside St. Vitus Cathedral, which is inside the Prague Castle complex. It’s a beautiful cathedral.

After visiting inside, we walked around it and then through the village area that surrounds the castle and took in some beautiful views. We then slowly made our way down the steep hill and back to the river. We parted ways as we got back to old town and they had flights to catch. This is when I made my way to a nearby hotel to meet up with my second Matt of the day, this time a friend and former colleague from OpenStack work.

Our first stop was for some lunch at U Fleků. It’s was founded in 1499 and claims to be the oldest brewery in Prague. They’ve really capitalized on this, it’s a prime tourist spot. They wander around with trays of beer, mead and becherovka. The food was nothing to write home about and it was a bit touristy for me, especially once the accordion began to play. Still, I’m glad we stopped by and I certainly enjoyed the beers, mead and the sip I stole of Matt’s becherovka. From there we took the long walk out to Vyšehrad. This is essentially the other castle site in the city, but it’s now a public park instead of a residence, though the cathedral and famous cemetery are in use for their intended purposes. We stopped at a cafe while wandering around to address some beer/mead sleepiness from lunch.

The views from here were beautiful as well, allowing for selfie opportunity!

While in Prague I also enjoyed tram watching. Had I bothered to figure out how to ride them I would have saved myself an incredible amount of walking. Maybe next time. Our journey back to our respective hotels did bring us back to the old town, this time in full weekend swing. It was packed with people. I was pretty eager to be on my way, though stopping for a filled Trdelnik was tempting, the whole neighborhood smelled like them.

My evening was spent with yet another friend who was in town for the Open Source Summit, Nigel! Nigel and I met through our involvement in the Ubuntu community. We’ve both gone our own ways at this point, but we’ve kept in touch and as we’ve both followed open infrastructure paths there’s frequent cause for us to chat. We met at my hotel and went to a nearby restaurant where I enjoyed both the dark and the light lager with lots of fun conversation.

That last night in Prague was an early one, as I had a 7AM flight Sunday morning to head off to Hamburg for my next adventure.

Lots more photos from Prague here:

pleia2 <![CDATA[All Things Open 2017]]> 2017-11-03T10:32:19Z 2017-11-03T10:32:19Z The goal of the trip down to Raleigh last week was to attend All Things Open. I attended back in 2014 and was eager to attend again, but was hampered by conflicting events. This year I committed to going, in spite of an unfortunate overlap with the Open Source Summit in Prague, which is important to me because MesosCon Europe happens on the Thursday and Friday of that week. This means I had to get on a plane immediately at the conclusion of All Things Open on Tuesday in order to make it to Prague by Wednesday night to begin my MesosCon EU activities. It was a long week.

But the week began in the most relaxing way possible as I stepped off the train in Raleigh Sunday night. Conference badge was picked up and I was off to a pre-conference social to say my first hellos to fellow conference-goers. I was quickly able to find some friends, but the social venue was quite busy and I was eager to find some dinner.

On the way out of the social I collected some folks and five of us ended up at an empanada and tequila restaurant not too far from the conference venue. Highlight of the night besides good company? They had pumpkin pie emapandas as a dessert. Heavenly.

The next morning the conference began with a series of keynotes. Todd Lewis, the grand architect of All Things Open and super nice guy, kicked everything off with welcomes and piles of gratitude. His personal kindness is one of the main reasons I love this conference so much. It’s also the largest open source conference on the east coast, this year drawing 3000 attendees, which is a new record for the event. That’s how I convince my employers to support the event year after year.

The first keynote came from Tim Yeaton of Red Hat who began his talk with a fun walk through a typical day that brought him to Starbucks, Chili’s, Target and then on to a United flight, all companies who are major users of open source and helping to drive the incredible growth throughout our industry. Jake Flomenberg of Accel joined us next to talk about the rise of open innovation, with companies like Intel and Goldman Sachs choosing open source software first and proprietary only as a last resort, a distinct change from what we were seeing a decade ago. He shared that there’s also a rise in venture capital backing of open source-driven companies who are driving innovation, it’s no longer just Red Hat making money in the world of open source business, others are following the path of Project, Product and Profit.

Danese Cooper, who has worn many hats in her lengthy open source career but is currently the Head of Open Source Software at Paypal, then graced the keynote stage. She began her talk by walking us thorough a history of why many people got involved with open source in the early days, recounting a story from Mark Shuttleworth. He explained his motivation behind investing his wealth in an open source project and company was because he cared and wanted to build something that mattered. The rest of us in open source software weren’t much different, even if we didn’t have millions to invest, my own passion drove my involvement through years of unpaid work. She shared that while many people today are getting involved for the very valid reason of being paid for their work, we can’t forget the lessons we learned along the way. To this end, we need to document our history and motivations so that future generations will know how we got to where we are, as well as mentor new people, serve as board members in open source organizations, and encourage good practices within our communities and company. She concluded by also stressing that we all must continue to follow our moral compass as we contribute, and that money to pay our mortgages will follow. That’s certainly true in my case.

Perhaps the most exciting keynote for me of the morning was from Sara Chipps, the CEO of Jewelbots. Jewelbots are tech-enabled friendship bracelets that have default modes to let you know when friends are nearby, but you can also extend them by writing C++ programs. She gave some history of the company, sharing that she wanted to build something for girls and noticed that they already had traditional friendship bracelets. By creating a tech-enabled one she discovered that the drive to extend the devices functionality-wise would “trick” the girls into having to write code. It worked, of the 10,000 units shipped, 44% of the users have extended them by writing C++ code, an incredible conversion rate. Finally, all the code for the devices has been made open source and is available on GitHub. Super inspiring keynote, it makes me so happy to see things like GoldieBlox and Jewelbots in the market for girls these days.

The final keynote of the day came from Kelsey Hightower of Google and Kubernetes fame. His talk was entirely demo-driven as he stressed that containers were not hype, they do simplify infrastructure by putting applications in nice little boxes. But that they aren’t a panacea, and while they help, we haven’t gotten to our future of flawless automation with them or any infrastructure technology. Much to our delight, he then was able to use Google-driven voice commands to show off not just a Kubernetes deployment on Google’s cloud, but an on the fly upgrade which he sees as the next big evolution in maintainability of large clusters.

With the keynotes behind us, the first talk of the day I attended was by Alicia Carr on Home Automation. The talk itself was a tour of the home automation devices she used in her day to day life, exploring how they interacted with each other (or not!) and her favorites on the market. She addressed the topic of security, admitting that we’re still in the early days of home automation and it’s not where it needs to be to be entirely safe and fool-proof for full, mainstream adoption, but that the companies in the space were aware of these limitations and working on it. Unfortunately there wasn’t much open source to this talk, some of the devices did allow you to code against them, and trigger tooling like IFTTT can allow you do do interesting things. I’d really love to see a talk about the home automation ecosystem of free software hackers that’s emerged, and the open source considerations therein. What devices are the most friendly to hacking? Which can be independently secured most easily?

The next talk I went to was John Mertic on “Accelerating Big Data Implementations for the Connected World” where he focused on the ODPi which seeks to strengthen the engagement model between upstream projects and end users. Five years ago the industry started adopting tooling like Hadoop to do big data processing at a large scale, but even then we weren’t at a place where we are today with the emergence of the Internet of Things and vastly more data coming in than we ever imagined. He shared that that the major challenge now is standardization. The options today for companies seeking to wrangle their data tend to be relying upon raw open source projects, which are frequently not productized enough for simple use, or being led by a vendor who may not have a standardization strategy in mind, instead steering you towards their version of productized piece of open source software. Companies need a resource to help fill in the gap there, and this is were the ODPi comes in.

After lunch I went to Corey Quinn’s talk on AWS cost control. He’s a fun and engaging speaker so I always enjoy going to his talks, but it was also incredibly informative. Though I’m not personally seeking to reduce AWS costs, it was fascinating to learn from his experience about what finance departments, C-level executives and company boards actually care about. Many consulting companies around AWS cost reduction focus on a series of technical strategies that can reduce cost, but his talk wasn’t about those, instead he focused on understanding what you’re running from a business perspective. It’s great to be efficient, but at the end of the day a large bill is actually fine if you can account for what you’re spending it on. Tracking workloads and having an itemized, defendable listings of costs goes much further than a painful cost reduction that knocks a few percents off of what is still an expensive black box as far as finance is concerned. Even better, when you do a project to start tracking your workloads, you’ll inevitably find services that aren’t being used and other extra fat that can be trimmed from your bill, and you do end up with the cost reduction too. Given the audience, he also shared the Open Guide to AWS and his own Last Week in AWS newsletter, both valuable resources to folks administrating AWS.

At 2:30 I gave my talk of the conference! I spoke again on The Open Sourcing of Infrastructure (slides here), taking the audience through the work we’ve done to choose open source as our infrastructure platform and how we can use those lessons moving forward in a world of proprietary cloud services. I gave a version of this talk back at FOSSCON in Philadelphia, but it’s one of those talks that’s always evolving based on audience feedback. The talk went well and afterwards I had a chat that will lead to some additional slides about security. Most specifically, there have been high profile security vulnerabilities in open source software and the major open source tooling that’s using it tends to have patches out within hours of disclosure. We don’t have that with proprietary tooling, in fact, the latest WPA vulnerability has left millions of proprietary devices without patches, and no indication to the customers about when patches will be available, assuming they will be and the devices and software are able to be upgraded at all.

The next talk I went to was from Elsie Phillips of CoreOS in the business track about building an selling an open core product. She walked through business models around training, support and consulting, sharing that these are still valuable avenues of making money, but that they generally weren’t enough and had very slim profit margins. There’s also a balancing act that must be made when you have open core software, making sure that the open source version is valuable and that the “Enterprise” version simply adds features, not that the open source version is restricted in some way. She stressed that the proprietary version should add polish like automation and other tooling that is not impossible to do on your own with the open source version, but saves the company engineering investment if it’s done for them out of the box. I happen to work for a company that uses this model, so while my open source heart bristles at some of the decisions made about what goes into the open source product, we do all have bills to pay and I often find myself in agreement as to where the line is drawn.

The final talk I attended of this first dway was by my Mesosphere colleague Alexander Rukletsov on Health Checking: A not-so-trivial task in the distributed containerized world. This talk drew from his experience with the overhaul of Apache Mesos health checks that occurred a year before. At first glance, it seems like doing health checks inside of containers would be easy since they live on a host that has access to them, but it turns out that it’s not. You have PID and networking namespaces to contend with that can turn even the most simple checks into something unexpectedly difficult. In a distributed system you need to consider the costs and benefits to how broadly you scope your checks, noting that a global scoping may be easier to set up, but you then have network overhead, latency and potential for duplication across your cluster. It was a valuable talk from an operations perspective allowing you to understand what goes into a check you’d generally consider “just an HTTP check” or similar.

All Things Open is a tiring conference and I was busy all day, so my day concluded with some nearby chicken and waffles before heading back to my room.

Tuesday opened with more keynotes! The keynotes began with a thoughtful and inspiring talk by Safia Abdalla who, among other things, called upon the audience to build friendships (not just mentorships) and to not just share code in our communities, but to share knowledge. Her talk was quickly followed up by one from Matt Asay, a familiar executive in open source circles and someone whose opinion I respect even if I don’t always agree with his conclusions. As he reviewed key open source successes in the corporate world he spoke to pragmatism as we push for more openness in our industry. He explained that convenience will typically win out and there’s always some amount of lock-in. There is victory even when building into proprietary clouds, they are more open than what we came from and most of what we’re building on top of that is still open source tooling. As someone who so strongly believes in open source as far down the stack as possible, I see his pragmatism argument, and even addressed it in my talk the day before (you can build against proprietary technology if you want, lots of companies do) but I’m glad for the industry that we have choice.

Burr Sutter joined the keynote stage next to give a very fun and accessible introduction to DevOps. He recounted stories of the improvements to code quality when developers shared in the responsibility in production, stressed the importance of on-demand infrastructure, automation and continuous integraton and deployment for fast, high quality development. John Papas was up next to talk about the open web, talking about some of the technology choices that should be considered when making modern websites, including ways to improve load time through priority loading of elements and searching code for module duplication and the presence of too many dependencies that could increase the size of the page being loaded. He concluded by asking us to be kind to one another, we may not see each other as we interact over coding sites and GitHub, but there are real people on the other end. The last keynote was from Jeff Atwood, the creator of StackOverflow and Discourse. His talk was a story through the creation of StackOverflow as a community and company, and how the more community-focused Discourse project came about. It was amusing to learn that having children so deeply impacted his approach to solving problems, or not solving problems. He learned that not every problem is actually in need of a solution, sometimes we just need to be human and empathetic to one another, and that’s enough. With StackOverflow the goal is very strictly about solving problems. With Discourse it’s about building a community and having fun.

The first session following the keynotes I attended was from Mark Voelker on Interoperable Clouds and How To Build (or Buy) Them. I knew of Mark from his work in the OpenStack community, and his talk centered around the work he’s done and lessons learned while participating in the “OpenStack Powered” program, where products can be certified through a testing suite to carry that label. It causes the OpenStack name means something, and ensures that is not haphazardly applied to products that replace pieces of OpenStack with their own tooling, run non-standard versions of the APIs or are otherwise not what the user may expect from something called OpenStack. He dove into some of the strategies the community uses for determining what should be tested and they iterate on the guidelines with every release, drawing from customers and product creators to learn what they should be including, excluding or changing. These lessons extend to what’s popping up in Kubernetes Certified and other programs in this space.

After grabbing lunch with one of my DevOps + open source compatriots, I met up with Spencer Krum, one of my former colleagues on the OpenStack Infrastructure team and current co-conspirator on open source infra work. He was doing live broadcasts for The Root Shell at ATO and I was invited to swing by to chat. It was fun, we talked about my current focus of containers and our work on open infrastructures. When we wrapped up I headed directly for the lightning talks that had already begun, making it in time to see Chris Short talk about DevOps and the joint official launch announcement with Jason Hibbets of the new DevOps Team, which I happily joined a few weeks ago. I also really loved the Lego hacking lightning talk from Jen Krieger, DYI electronics hacking with Lego looks like a lot of fun and she showed off some neat projects.

Now, in spite of leaving much of my direct operations work behind when I changed jobs last year, Jenkins still looms large in my work today as I am working on a CI/CD demo and frequently give talks about how containerization can be used in this space. As such, I went to a talk by Kristin Whetstone on “7 Habits of Highly Scalable Jenkins Administrators”. The habits included using the latest LTS version of Jenkins, using Jenkins Pipeline and container-based plugins for more self-service job deployments, parallelization of tests and participation in the Jenkins community. What I really enjoyed about this talk was that she not only would describe the problem space of each of these, but gave links to the specific plugins that would help you achieve your goal with them. The day was getting long as I went with the fun choice for the next session, and joined Gareth Greenaway in his “What the FOSS Community Can Learn from 80s Television” talk. It was a funny trip down 80s television memory lane with open source lessons, including: the importance of working as a team, ignoring MacGyver’s failure to document his solutions, improving the bus factor, reducing the impostor feels by new contributors as Sam in Quantum Leap perpetually felt, and the importance of not simply creating new projects (characters) or forks (spin-offs) without a good reason.

In the late afternoon I met up with my friend Laura over at the booth she was running. The plan? A photo of the Ubuntu gurus we knew at the conference. Success! Not many of us are working on Ubuntu any longer, but the friendships we made in that project sure have lasted.

Our connection? Ubuntu!

With that, we were just one talk away from the end of the conference, and my need to get to the airport. The final talk I attended was by my friend and former colleague on Ubuntu work, Michael Hall. His talk had the interesting name of “The Revolution Will Not Be Distributed” which you quickly learn is not about distributed infrastructure, but instead about the way that software itself is delivered. One of my first contributions to an open source project was packaging in Debian, and it was hard. I had a mentor and throughout my work in Debian and Ubuntu, even as I helped others with packaging, I was always consulting the documentation and asking for help from other people about the proper and expected way to do various tasks with Debian packages. At the time I hadn’t thought much about how this wouldn’t scale, and I was right along side everyone else who worked on various efforts to teach more people packaging and to get software creators to package their software for distributions. This included various efforts that the community team at Canonical embarked on to get more software packaged, none of which resulted in a significant, sustained increase in the number of packages or packagers in the community.

Michael ran the numbers for us. There are about three million applications in the Android marketplace. The Ubuntu desktop has about three thousand. You can definitely argue that the Ubuntu includes high quality applications that build a firm and sufficient foundation for a workable desktop, but we all know that there are gaps, and software we install outside of the standard repositories, whether it’s by using a PPA, install script, or continuing to compile it by ourselves. With this baseline gap, along with statistics about the current number of package maintainers per package throughout various distributions, he calculated that the number of maintainers and people-hours involved to get to Android was astronomical. Even to get a respectable fraction was not within reach. Instead of continuing to toil in the world of distributions with their own package formats, he looked to what is already happening with formats like Flatpak and Snappy, making packages that are simpler to make and portable (especially with FlatHub and the Snap Store), as well as the increased adoption of containers. His vision for the future of distributions was less about packaging of software itself into DEBs and RPMs, and more focus on integration of this software with the desktop environment for a more pleasant user experience. It was an interesting talk, and certainly gave me a lot to think about with regard to our approach to the distribution of software.

More photos from the conference here:

It was then off to the airport! I cabbed it over with my friend Stephen, Michael and David, and the latter two ended up being on the same flight as me. Even more amusing, our connections in Charlotte (not a small airport!) were leaving out of a trio of gates right next to each other. It was a lovely way to conclude my ATO adventure.

Huge thanks to Todd and the team who put on All Things Open. It’s a wonderful conference and I was happy to participate again this year.

pleia2 <![CDATA[The Carolinian]]> 2017-10-31T18:43:09Z 2017-10-31T17:48:45Z On October 22nd I took another US train trip. Like my recent journey with MJ on the Coast Starlight down to Los Angeles, this trip on The Carolinian was a day trip to get me to a conference. This time I went with David from Philadelphia to Raleigh for All Things Open, the train left from 30th Street Station in Philadelphia around 9AM. With the exception of a trip up to NYC from Philly back in June, my big Amtrak trips have primarily been in the double-decker trains that included sleeper cars. This one was not, the top class was Business, which we took, and there was a cafe car, but no formal dining car. I did notice that the Business class car seemed to have new seats though, which was a nice change from all the old equipment I’ve experienced this year on other trains.

Business class cabin, our spot for the eight hours to Raleigh

The early portions of the journey traversed familiar stomping grounds for me, with stops in Wilmington, Baltimore and Washington DC. This was also the relatively quick portion of the trip on the high Amtrak traffic northeast corridor, taken with an electric locomotive.

Coming into Baltimore

Washington DC gave us about a half hour pause for passengers and swapping of the locomotive to a diesel one. Having done this route before, David clued me into the fact that we could head to the front of the train to see this. The power shut off in the trains and we hopped off, the electric engine was detatched and within about 15 minutes the diesel engine came rolling down the tracks to take us the rest of the way.

Electric engine detatched!

Plus, while we waited we got our train selfie!

From there, it was on to Virginia, for hours. It was nice to see the changing foliage in Virginia, somehow Philadelphia mostly consisted of green and brown-leaved trees when I was in town, probably caused by an unusually warm autumn that has confused the trees. The train also passed through fields and farms, through the woods and various small towns that had quick train stops. What I didn’t quite expect on this journey was the small towns, especially in southern Virginia, that had very little to show for in their downtowns. What was clearly once thriving, was now a series of boarded up and closed businesses. Of course I’d heard of the demise of the rural downtown, but being big city dweller that I am, seeing it, even from the window of a train, made an impact. A real estate app on my phone also kept alerting me to nearby home prices, which dipped well below an average of $100k in these areas, aside from the notable exception of an 18th century plantation estate that was going for $2.5 million.

Virginia stretched on for hours, with lots of nice things to see!

Lunch fell on this long stretch in Virginia. The cafe car was equipped with a microwave, which all hot food was unceremoniously prepared in. I’ve not generally been impressed with train food even in a formal dining car, and this was a step below that. I enjoyed my cookie though. There were also tables so we didn’t need to bring our meals back to our seats and were able to enjoy a slight change of scenery. I do admit missing the vista-type viewing cars from some of my earlier Amtrak journeys. Still, the company was good and I was able to get a bunch of reading in during my trip. I also probably spent two hours just staring out the window and taking advantage of the relaxation time that a train ride provides over air travel.

The afternoon brought us to the Roanoke River crossing, in North Carolina! The train enters the state near in the northeast and ends in Charlotte, which means that after going distinctly south for most of the journey, the train began making its way west after the stop in Wilson, NC.

Roanoke River crossing, North Carolina

A stop in Selma was next, and shortly after 5PM and slightly behind schedule the train arrived in Raleigh. The conference venue was just a couple blocks from the station, so a quick walk got us there for conference check-in and then off to the pre-conference social. No post-plane de-stress time needed, I was totally relaxed. As such, it was nice to be able to jump right into visiting with my open source brethren. Bonus: I shared the journey on social media so the most common non-work exchange I had at the conference was about how I had taken the train down, and often sharing details about how they could have done the same.

Off the train in Raleigh!

I’ve called 2017 my year of Amtrak, and though a jaunt up to NYC around the holidays with MJ is not out of the question, this was probably my final long-haul trip of the year. It won’t be my last of my lifetime though, I’ve quite enjoyed seeing the country by train this year. Though trains in the US leave something to be desired, especially when it comes to food, it’s a peaceful and comfortable way to travel.

More photos from the trip are here:

pleia2 <![CDATA[Fires and fleeing]]> 2017-10-21T21:48:51Z 2017-10-21T21:48:51Z After the Grace Hopper Celebration of Women in Computing in Orlando recently my friends Michael and Michelle were kind enough to put me up for a night in their nearby home. They served me a home-cooked meal and set up space for me to crash in one of the kids’ rooms. I got to visit with their kids, meet their trio of big dogs, but mostly catch up with them and enjoy some down time. It was one of the most relaxing respites I’ve had in months. Thanks again, Michelle and Michael!

From their place it was off to the airport on Saturday afternoon. After a quick layover in Miami, I was on my way back to San Francisco, putting me at home just after midnight. Unfortunately my luggage did not join me. All the luggage from the Orlando flight missed the Miami connection, leaving a third of a plane full of disappointed travelers. Thankfully I had no where else to be but home, so a luggage delay was no big deal, they delivered it to my home the next day.

Following this trip, I should have had nearly two weeks in town to prepare for three weeks of travel abroad, but nearby wildfires had different plans for me. By Sunday night smoke from wildfires in Napa and Sonoma had descended upon the city. It was fun at first. We didn’t realize how close the wildfires were getting to homes and business up in Napa and Sonoma and it was a bit like camping, with the wood smoke smell drifting in through the windows. By Monday morning the reports had started coming in from up north about how bad the fires were, people were losing their homes, some historic buildings were lost, wineries we frequented were taking precautionary measures. At home I was clearly coughing, and come Tuesday my chest was tight and sore from the coughing. This was about when we all learned about and could start tracking the air quality. With the levels remaining at “Unhealthy” on Wednesday I finally heeded the advisories and closed the windows. This made our condo quite warm, getting up to the mid-80s, as we lack air conditioning and usually depend upon the breeze from the windows. This was when MJ first suggested I fly back east to spend time at the townhouse in Philly.

Still, I spent the week as I usually would. MJ and I had some dinners together, Caligula and I chilled after work, one evening I made banana bread when I realized the bananas I had delivered with groceries earlier in the week were a bit more ripe than I had expected. In spite all the tragedy in the reports coming down from wine country and being cooped up, the smoke hanging in the air did make for some fascinating views of the sun and sky. Our sunsets were dripping with pale red, making some doomsday shadows on our walls.

I wasn’t the only one taking photos of the red-tinted sun on Friday evening when MJ and I went to the Synagogue for Shabbat services and Simchat Torah.

Simchat Torah! Jewish congregations read the Torah from scrolls made of parchment and hand written by a scribe in Hebrew. This means there’s a beginning and end of the scroll, and with the Jewish new year behind us, Simchat Torah is when you read the final passage at the end, and then the scroll has to be wound back to the beginning. To celebrate this, our synagogue took the opportunity to fully unravel the scroll and our Rabbi gave us a tour of it as we all held it around the sanctuary. It was a pretty special thing to be a part of, I’m glad we both were able to attend, even if it was earlier on a Friday evening than we tend to go to events, due to proximity to work timing-wise.

That evening after services was when I decided that going back east was not a crazy idea. It actually just meant leaving five days earlier than I had planned on, since I was already planning on going out a day ahead of my trip to Raleigh for the All Things Open conference. MJ helped me find a good itinerary and rebook my ticket for a flight out of San Jose on Sunday night instead of the following Friday.

Saturday night we fled to a nearby hotel to enjoy the climate control which would give my lungs a break until I flew out. Plus, it was a relaxing mini-getaway that allowed us to enjoy their executive lounge, where they had chocolate mousse in chocolate cups!

Plus the views were nice. The sunset was quite a bit less apocalyptic by Saturday evening. In fact, I started reconsidering my trip back east, had I over-reacted?

It turns out I probably hadn’t over-reacted. Though there was some clearing of the air on Sunday and we were able to spend some time outdoors, but fires and winds picked up early in the week to make San Francisco smokey again. I did feel a bit bad for leaving though, I’d miss being home, my husband, my cat, and I wasn’t quite prepared to leave so soon on a trip that was already taking me away for three weeks. Taking care of my lungs was important and we have the townhouse in Philadelphia where I can stay without needing to make prior arrangements or costing us anything more. Childhood asthma seems to have done a number on my lungs, but thankfully MJ isn’t prone respiratory issues like I am, so he seems to have toughed out the smokey conditions with no ill effects, like most other people I know in the city.

Plus, it ended up working out really well for work. Most of my team spent most of the week in Europe, and with our MesosCon Europe planning I had a few 7AM pacific meetings, which is a much more tolerable 10AM eastern time. It also gave us the opportunity to get the sprinkler system in the townhouse inspected on schedule and get some work done on the sprinkler pipe that we had talked about to make room for a future water softener. It’s also my favorite time of year in Philadelphia, I love autumn. The temperature and low humidity was such that I could open the windows and enjoy the fresh air for most of the day. I also lucked out precipitation-wise and ended up with beautiful, clear skies all week.

I was pretty anti-social during my visit though. I took a ride share from the airport to the townhouse upon my arrival on Monday morning and then spent Monday and Tuesday at the townhouse on my own. Wednesday and Thursday evenings I finally made it out to visit with friends, which I’m finally also doing today. Today is my last day in Philadelphia though, tomorrow morning at 9AM I’ll be on am Amtrak train with David down to Raleigh for a Sunday evening arrival, hopefully coming in with enough time to go to the opening social for All Things Open. I have a lot of open source friends and colleagues attending the conference, so I’m really looking forward to it, even if I do have to depart Tuesday evening with enough time to catch a 7:40PM flight that takes me to Charlotte, London and finally Prague for MesosCon Europe. I have a few busy weeks ahead of me, I’m not entirely sad about taking the time to myself to catch up on personal projects.

pleia2 <![CDATA[Grace Hopper Celebration of Women in Computing 2017]]> 2017-10-11T00:00:24Z 2017-10-10T21:20:44Z Last week I had the pleasure of attending the Grace Hopper Celebration of Women in Computing for the second time. The first was back in 2015 when I spoke during the open source track. This year I was attending to represent Mesosphere at our booth in the career fair, we were a silver sponsor of the event.

I arrived on Tuesday afternoon and met up with my colleagues Amita Ekbote and Susan X. Huynh. I was able to get checked in for the conference, have some lunch together and visit the booth in the expo hall to confirm our delivery of stickers and t-shirts had arrived. Since we work on different teams at Mesosphere, it was really nice to spend the afternoon together talking about our work, the company, and thoughts around diversity initiatives that were inspired by our attendance at the conference.

GHC is a very career-driven event, which is showcased in what is not an expo hall, but a career fair. Companies from around the world pour a lot of money on event sponsorships and booth setups to attract talent to their booths, whether in the form of internships or full time employment. The conference had 18,000 attendees, mostly women, and that’s a big pool to draw from.

This was my first time doing booth duty with this kind of focus, but I thoroughly enjoyed it. As a startup, our booth was modest but we have a great logo and the traffic at the booth was brisk throughout the event. I met a lot of women who are interested in security, which I was very glad to see, since the field of security in technology is quite male-dominated. More applicable to our work, I did meet several women who had experience with distributed systems, some of whom had even worked with Apache Mesos before. There were a few times when I got to go off into the weeds a bit, geeking out with an engineer who came to our booth and was enamored with infrastructure work.

Margaret Sy, me, Shira Oatis, Amita Ekbote and Susan X. Huynh, credit to Shira for the sticker display!

There’s something incredibly special for me about getting to work and geek out with other women. As a woman working in the intersection of infrastructure and open source it’s rare for me to be able to. Throughout my entire career, the vast majority of my colleagues have been men. While I’m very happy with the men I’ve worked with, there’s a different connection I have to other women that I really enjoy when I can find it.

And one afternoon I also spent some time with my droid buddy R2-D2. Thanks to the Disney booth!

Now I wasn’t able to make it to any of the sessions at the conference, but I did watch the keynotes. They showcased an incredible lineup of women in technology. On Wednesday morning at the conference kickoff I really enjoyed hearing the funny anecdotes from Melinda Gates as she walked us through her teenage love for her Apple III (“this isn’t Bill’s favorite story”) and the failure of Microsoft Bob, a project that she was a project manager for. She concluded her talk by imploring us and the industry to examine and appreciate non-standard entrances to computing. I loved computers as a teen, so did Melinda, we could geek out about old computers we had all night, but we’re the ones who succeeded. Not every woman, or person, comes into computing that way, but it is the path we focus on and venerate. We need to embrace different paths to careers in tech and make sure we’re reaching the women who don’t match our stereotypes.

The speaker I was most excited about on Thursday was Debbie Sterling, the Founder and CEO of GoldieBlox. Thanks to the grape vine that is the women in tech community, I knew about and I was one of the early believers in GoldieBlox. I voted in the campaign to get them an ad in the Super Bowl which catapulted their success. Hearing her story of following her passion and incredible resistance and rejection she faced before finally succeeding to get VC and funding through a Kickstarter was inspiring. The whole idea of GoldieBlox was also something I believe in, we need more toys for girls that allow them to create and build, and not be read-only participants in the pink aisle.

The other thing that jumped out at me was how incredibly accomplished every single one of the keynote speakers was. At most conferences I attend the keynote speaker may have written a book or founded an organization, but every woman up on the keynote stage as the days progressed had several noteworthy accomplishments under her belt, and I’d never heard of many of these women. Between the keynote speakers and the sold out audience, anyone attending this conference must walk away feeling like we have an incredible talent pool of women in tech in this country.

Theme-wise, there were a couple that jumped out of me. The first was the diversity of their speaker line-up, and how many of them came from less affluent communities. There were women from Ghana, Ecuador, Rwanda, Mexico and China, many of whom immigrated to the US and all of whom have overcome discouragement and adversity to push ahead and achieve incredible success. I’m sure I wasn’t the only one in the audience who was then doing a great deal of self-reflection. We struggle with and are exhausted by microaggressions and harassment throughout our career, but these women have had to overcome so much more and still believe that it was worth it. Improvements of the “first world problem” type definitely still need to be made since they do still take away from the time we have to work and succeed, and still cause women to leave in significantly higher rates than men, but perspective I gained from the experiences of these women was important for me. Keep going, it’s worth it to be players in industry that builds our future.

The second theme was one in the direction of Artificial Intelligence (AI) and Machine Learning (ML). Both of these fields are already having an impact on our lives today, and will to a much greater extent in the future. From Dr. Fei-Fei Li we got clear examples of how “machine values are human values” and that even the most well-intentioned developer still has unconscious bias that can lead to problems where they have blind spots due to their background, gender, race and more. The AI and ML fields need diversity of engineers, data sets and people who ask real human problems when crafting technological solutions.

Finally, throughout the keynotes there were projects and initiatives that speakers mentioned, the following are a handful that stood out for me:

You can watch the keynotes from each day by following these links, and details about the speakers can be found here (where I got these links from):

I had several great meals with my colleagues, but no conference would be complete without a party, and Women Techmakers put on a great one over at the Epcot World Showcase. From 7 to 9:30 we enjoyed small plates and drinks, a great pro-women DJ, a bunch of activities brought in by Google, and even a Waymo self-driving car! I got to check out the new Pixel 2 and Pixel 2 XL, which was great since I think the Pixel 2 XL will be my next phone and getting to play with one was really nice. I also stood in line with my colleague Margaret to get “selfie stickers” printed by Google Allo, which took a picture of you and then generated a cartoon image from it (mine are here).

At 9:30 the park closed to the general public and we had 90 minutes to explore it on our own! The countries in the World Showcase were all closed, and no wine and food festival for us, but we did get to go on the iconic Spaceship Earth ride and Soarin’: Around the World before the evening wound down. Unfortunately my sprained ankle wasn’t a huge fan of this late night, but I’m glad I pushed myself a little so I could take advantage of the evening.

The conference concluded on Friday with a couple final shifts at our booth, during which my boss, and company co-founder, Ben Hindman joined us at the conference and the booth for the day to meet with folks who came by. The four of us remaining then took the opportunity to swing by the eBay booth to pose in front of a great quote from Grace Hopper, “The most dangerous phrase in the language is, ‘We’ve always done it this way.'”

Thanks to Shira for sharing this photo (source)

I decided to depart before the afternoon keynotes in order to avoid rush hour traffic and the rush on cars for hire, but I did watch them later. I was really amused by Ayanna Howard’s keynote where she shared that her studies show that people trust robots, even when they’re not working properly, so there needs to be caution and thoughtfulness involved as we program them. I also had a particular fondness for Maureen Fan’s keynote where she talked about her company, Baobab Studios, and her livelong love for animation. I share this love, and for the same reasons of hope, dreams and taking you to faraway and imaginary places.

In all, I enjoyed the conference and all the people I had the opportunity to meet. It’s nice to be reminded now and then that there are a lot of women working in tech, even if the percentages aren’t where we want them to be. We’re here, and we belong.

A few more photos from GHC here:

pleia2 <![CDATA[A few days in September]]> 2017-10-03T01:03:20Z 2017-10-03T01:03:20Z I spent about half of September traveling, but between trips to Dublin and Los Angeles I was able to spend a little time at home in San Francisco to get kitty snuggles, observe the Jewish High Holidays with MJ, speak at a local conference, get my ankle looked at and celebrate my birthday. I’ve been busy.

We spent Tuesday through Friday at home between Dublin and Los Angeles. I was able to catch up with a bunch of work post-travel and went into the office on Wednesday, as I do when I’m in town. I ducked out of my office a little early to head over to the Microsoft office in San Francisco. It was just before MesosCon and I wanted to meet up with my fellow DevOps track lead to make sure we were all set. We enjoyed some coffee and went back to to the office to hack on a few things before evening crept up and I had to make my way home. It was fun, I’m not much of an office person but I do enjoy visiting other people’s offices to work now and then.

The condo has been hot this month. Thankfully we missed the record-breaking 100+ temperatures, but even when it’s 85 degrees outside, our 11th floor unit gets very little breeze and it heats up quickly. A small standing A/C unit helps a little, but it’s mostly for taking the edge off the worst of the heat. The heat got even worse one night when our old toaster oven caught on fire. I quickly learned how much of a pain it is to clean up after a dry chemical fire extinguisher, but no damage was done. We picked up a new toaster oven after I got back from LA, and Caligula was pretty excited because he got to enjoy the toaster oven box until I put it away.

I returned from LA just in time for MJ to head out of town the next day for work, just a few days before Rosh Hashanah. But I did have him back in time to celebrate together on Thursday the 21st. We typically go to the evening services and the daytime services on Rosh Hashanah and this year was no different. The daytime services concluded with a Tashlich at Crissy Field, near the Golden Gate Bridge and it was a beautiful day for it.

I also attempted to make a round challah! It was more square than round, but I also did an experiment to make it with honey and it turned out really nice and moist. So next year I know to still make it with honey, but work harder on actually making it round.

The day after a bunch of standing and walking for Rosh Hashanah, I decided I needed to see a doctor about my ankle. While I was at Newgrange in Dublin I twisted it pretty badly. Walking on it in Los Angeles through tar pits and two conferences didn’t do me any favors. I think standing on it during services was the last straw, it was hurting that day I went to the doctor more than it had when I first injured myself three weeks before. That trip to the doctor led me to get x-rays, and x-rays hinted at a break which sent me to an orthopedist the following Monday. Of course that meant I spent the whole weekend worried about whether I’d broken my foot. Ugh.

Thankfully there was no break, but I did get to discover the open source Aeskulap DICOM Viewer and so I got to see pictures of my bones! I was instructed to rest my ankle, since I had clearly done more damage to it. It kept me working from home all week, except for a local conference on Wednesday where I was giving a talk. Fortunately, the talk was only fifteen minutes long and after spending a bit of time at our booth, I was able to grab a ride back home for some more ice and Aleve.

The talk was a shortened version of my Day 2 Operations talk. Given the 15 minutes I had, I was able to take a more playful spin as I explored four myths around operations and containers:

  1. Containers will solve all your problems! (they solve a lot of problems, but create new ones)
  2. Green fields! (no, you still often have complicated legacy infrastructure)
  3. Everything is already automated! (it’s not, but most things have APIs now to hook into existing tooling, nice!)
  4. No more planning! (easier to deploy, but make time to add in logging, metrics, maintenance plans)

I enjoyed the 15 minute format, it felt a bit rushed, but it also caused me to yank out a lot of slides that were “here’s a list of technologies that do this” and focus more on the overall message. Though I always do feel weird telling ops folks they still need logging and metrics in this world of containers, the message is always well-received as they bring it back to their workplaces to argue for the time to build better a more maintainable system. Slides here.

Thanks to Rachel Dines for taking a photo during my talk! source

I wish I had been able to spend more time at the conference though, it was the Sysdig Camp-Con-World-Fest-Summit and there were a number of generalist container and metrics talks that would have been interesting to see. It was nice to meet up with fellow speakers at a speaker dinner at The Slanted Door in Ferry Building the Monday before the event though, we even got to eat clouds for dessert! Alas, foot-wise I needed to take care of myself.

That night I had to cancel some of my other plans as well, but I did pay for a ticket to see Ellen Pao speak on Resetting Silicon Valley so that made my priority list. In spite of my sore ankle, I’m glad I went. Stories of systematic sexism are never easy to hear, but I really enjoyed hearing about Project Include and their 87 Recommendations for companies looking to increase diversity and inclusiveness. The interview with Pao gave me hope for the future of diversity and inclusiveness, as not only projects like these are popping up, but there’s a much stronger understanding of and push back against the now acknowledged problems of sexism, racism, ageism and more in Silicon Valley.

Friday was my 36th birthday! The chaos of travel lately and the evening of my birthday landing on Yom Kippur made it so my birthday was considerably more low key than it’s been in previous years. We skipped a proposed getaway up to Tahoe in favor of a much-needed weekend at home, and went across the street to Fogo de Chao for a wonderful, filling birthday meal before the sun set and we were off to Kol Nidre services and into a day long fast. My best friend did send me flowers though, which was an incredibly sweet surprise. I also accepted the free dessert offered at Fogo, having been interested in their grilled pineapple dessert for some time but usually too full at the end of my meal to even think about dessert. They decorated accordingly!

Yesterday morning I met up with my friend Laura for breakfast. She’s in town for the huge Oracle conference that’s been under construction at Moscone for several days. One of my colleagues is speaking at the conference, but I’m off to Orlando tonight to help out with the Mesosphere booth at the Grace Hopper Celebration of Women in Computing. We only get to catch up in person a couple times a year, and now that my role is on the developer advocacy side, I have a ton to learn from her experiences doing this professionally for several years. She shared tips about working with a budget, “on call” rotations for covering social media, Slack and other community-driven resources and goal setting. In spite of doing this on a volunteer basis for various communities over the years, there’s a lot to learn once you’re in a position when you’re being paid to help engage a community, even if you’re passionate and excited about your work. I have much to think about after our conversation and I look forward to sharing it with the rest of my team.

Grace Hopper will take up the rest of my week. On Friday after the event concludes I’ll be staying with some friends for the night, one of whom is another developer advocate who I met through the Ubuntu community. No doubt we’ll have lots to talk about and catch up on. I’m flying home Saturday night, and staying in San Francisco for just under two weeks before I’m off again. Next adventure? All Things Open, MesosCon in Prague, various Meetups in Germany and then CubaConf. I’ve got my month cut out for me! I’m also pleased to have booked our trip back to Philadelphia over Thanksgiving and gotten kitty care worked out for that. It’ll be nice to have some family time and get back to the townhouse for a week or so.

pleia2 <![CDATA[Sunday at La Brea Tar Pits and Venice Beach]]> 2017-10-02T15:29:27Z 2017-10-02T15:29:27Z Our journey on the Coast Starlight last month put us down at Union Station in Los Angeles and we grabbed a rideshare over to our hotel for the night. Our hotel had a robot, but unfortunately it was out of service. Poor robot.

Sunday was our tourist day! MJ had to fly back home to get to work on Monday, and I was changing hotels to attend the Open Source Summit Monday morning. The main thing on my agenda was finally getting to see La Brea tar pits. I’d been down to Los Angeles several times, but it’s always been on business, and since it’s so close to San Francisco I often don’t extend my trip to do much visiting. This was an opportunity to finally see the tar pits, and actually do it together!

First we decided to have a leisurely morning. Enjoy breakfast outside, grab a local microbrew and spend an hour at the hotel pool. We left ourselves a bit of time to shower and pack up before checking out and finally making it out to the tar pits.

We arrived in the late afternoon, which gave us enough time to visit the museum before it closed. The museum is full of ice age fossils, some of the more impressive being the mammoths and sabre-toothed cats.

Formerly know as an American lion, I learned that it’s actually more closely related to the modern jaguar, sabre-toothed cat behind it

I’d also seen the wall of 404 dire wolf skulls in documentaries about the tar pits before, but seeing it in person was impressive.

As the museum hours wound down, we decided to walk around the tar pits themselves for a bit. I think the order we did things in was a mistake. It would have been more enjoyable to see the exhibits with a bit more context than the inside of the museum itself provided. The museum is on the older side and the exhibits themselves hadn’t been refreshed much over the years. They seemed to be more on the side of impressing kids with a thin layer of facts, some of which were out-dated, rather than getting into the history of the tar pits themselves. I did enjoy it because the somewhat campy feel of the museum was appealing to me, but I do like good modern museums too.

Now the signs outside at all the pits themselves filled the gap the inside of the museum left. If I were to do it again, I would have started at the side of the park where the Observation Pit is.

Signs explained that the observation pit was the oldest museum on site, housing a tar pit and still actively giving tours there in association with the museum, though they were over by the time we got there. From there a walk around the tar pits brings you close to a bunch of signs, talking about the history of the pits, how they were created and re-discovered, answering questions.

The outdoor pit area included the iconic Columbian mammoth struggling to free from her tar pit fate while two other mammoths look on, and then a sign next to it debunking the scene as it’s depicted. I thought this was really good. The sculptures were put in decades ago and are now famous along with the site so it would be sad to see them go, but the outdated portrayal is unfortunate.

The walk around the pits also brought us to Project 23. Millions of fossils were found as they were building the new Los Angeles County Museum of Art nearby. Instead of halting development for years, huge chunks of tar pits were removed to be examined later and building continued on the site. The boxes were put in their current place in 2008 and the project has been ongoing ever since.

After our tar pit adventure it was time to find dinner. We headed out to Venice Beach. MJ had been with colleagues before, but it was all new to me. We ended up at L.A. Gastronomy where I had a lovely dinner of yellowfin poke on rice with seaweed, sesame, caviar, daikon and scallions, along with an appetizer of their short rib poutine.

After dinner we still had a bit of time to walk around the pier at the beach nearby to watch the sunset before MJ’s flight. We even saw some lightning off the coast, which is not the most common thing on the coast of California.

With the sunset concluding our evening, it was back to the hotel to pick up our luggage and go our separate ways. It was a short getaway, but it was relaxing and I really enjoyed finally getting to see the tar pits.

pleia2 <![CDATA[The Coast Starlight]]> 2017-10-02T05:07:57Z 2017-10-02T05:07:57Z I’ve wanted to take a journey on Amtrak’s Coast Starlight for years. Alas, timing was always a challenge. It’s so hard to justify spending 11+ hours on a train when a short flight will suffice, but 2017 has been my year of trains. Over Memorial Day MJ and I took the California Zephyr and Capitol Limited to get across the country and this month I’ll be hopping on the Carolinian with a close friend, and fellow train adventurer, from Philadelphia to Raleigh.

We finally decided to take the Coast Starlight down to Los Angeles a couple weeks ago to get me to the Open Source Summit and MesosCon.

Now I’m going to pause for a moment to talk about my feelings about Amtrak. It’s the only long-haul passenger service in the United States, and I love trains. It makes it my de facto home train so I have a special fondness for it. But Amtrak is expensive, often more so than flying, and the cars are extremely dated. The food even in the dining cars isn’t great, and there is no variation in the menu between routes that I’ve seen thus far. They serve a fair amount of pork, which excludes much of the menu for me and making the menu even smaller. I can’t imagine riding on it as a vegetarian unless you packed your own food. The air conditioning leaves much to be desired and even if you get on a train that advertises having WiFi, your best bet is to assume it’s not working. The WiFi situation often isn’t their fault, coverage is simply spotty through some of their service area, which leads me to another thing out of their control: the United States is big and taking a train that typically must travel below 79 MPH is going to be slow.

Still, it’s my train and I enjoy taking trains. The views, the disconnection, the dull roar of the rails. I can relax, get some reading done or chat with my fellow passengers in the lounge car.

MJ and I departed early on a Saturday morning to begin our trip at an Amtrak bus station near the Embarcadero here in San Francisco. Unfortunately our train was delayed, as I hear often happens with this route, so after the bus dropped us off at Jack London Square station we had a couple of hours to kill while we waited for the train. It did finally arrive though!

In spite of not doing an overnight this time, we sprung for the Roomette so we could have have private space to retreat to during our 11 hours on the train. This class of room got us access to the famous Pacific Parlour Car. The status of the six Pacific Parlour style cars is tracked on, explaining that five of them are in operation by Amtrak on the Coast Starlight on train numbers 11 southbound and 14 northbound. These are pretty special car in the Amtrak fleet, they’re Hi-Level lounge cars by The Budd Company in 1956 for the Santa Fe Railway, not Amtrak. They’re also shorter than the other cars, which you notice a little when walking between the cars but it’s definitely noticeable from the outside of the train. As for the features of the car itself, this page has a photo tour that takes you through the bar area, seating area, dining area and downstairs to the movie theater.

The Parlour Car

In the afternoon the parlour car also hosted a wine tasting, but while it was fun to move around a bit and have a few sips of wine, the wine snob in me was not terribly impressed with the selection. I did manage to have a Stone IPA during dinner though, which somewhat to make up for the lackluster showing from the wines.

The downstairs movie theater was pretty cool, unfortunately they weren’t showing any movies. When I asked about the movie theater one of the staff members said the royalties were too expensive so they currently couldn’t show any. I’ve also read on the forums that they’ve struggled with the technology aspect of it, switching to DVDs when satellite feeds wouldn’t suffice. I couldn’t stop thinking about this for the rest of the trip. What an opportunity for someone interested in developing a Creative Commons program! Old movies! Modern, CC-licensed cartoons! So many great options! Maybe some folks would be disappointed that it wasn’t known blockbusters, but it’s quality content and certainly better than nothing. Develop a reliable SSD-based box to stick in there to play the content on the TV and then you get to avoid DVD and satellite issues. I actually now want to be the crazy tech lady who runs the movies on the Coast Starlight.

All that said, I probably wouldn’t have watched a movie during the trip. Hah! As my first journey on the Coast Starlight I was eager to see the scenery down the whole route.

Down through the bay area we had the predicable salt marshes, a group of hikers here and there. As we went further south we passed through a lot of farm land. Here and there the backdrop of mountains made for some views worth seeing.

I also managed to get a panoramic photo as the train navigated the horseshoe curve at Cuesta Pass. Zoom in on the photo, you’ll see both the front and end of the train! Our car was near the middle of the train, and it takes a few seconds to take the panorama, making the following photo possible.

And then just south of San Luis Obispo we got to the amazing part of the trip. The train heads west and runs along the coast line. We enjoyed dinner as we looked out over the cliffs and beaches.

No joke, RVs, surfers and palm trees ruled down here.

The sun set as the clock ticked past 8PM and the rest of the trip was taken in darkness as we passed through Oxnard and Burbank. It was interesting to learn that the Burbank airport is attached to the Amtrak station, which is a rare thing here in the US, but obviously makes a lot of sense. We got into Union Station in Los Angeles just after 9:30PM on Saturday evening. Onward to our Sunday of adventures in Los Angeles!

Sub-optimal air-conditioning aside, I really enjoyed the trip down and I’m glad we finally took the time to do it. More photos from the trip down here:

pleia2 <![CDATA[Galway and the Cliffs of Moher]]> 2017-09-29T15:28:35Z 2017-09-29T15:28:35Z Our last day in beautiful Ireland this year was spent on the west coast. MJ had been out there, but it was the first time for me. He’d also driven on the left side of the road before, so when it came to renting a car, he was the driver. That morning we left the hotel early to pick up or lovely BMW rental car and make our way out to Galway!

The drive took about two and a half hours, we found parking and quickly discovered we were in the midst of a city that was extremely excited about the hurling championship game that Galway was playing in. I knew nothing about the game when we walked into The Quays for lunch. I still know very little about it, but we had a lovely lunch with one of MJ’s colleagues and his wife who also happened to be on that side of the country for the weekend. Plus, I got my beloved fresh north Atlantic oysters.

After lunch the sun came out and we walked around town a bit, walking right past Lynch’s Castle (a medieval “town house”) without realizing it as it’s a bank these days. We then walked past a couple of old churches before making our way down to the Spanish Arch and then back to the car. The morning had been rainy in Dublin, but the stunningly blue sky that met us after we saw the arch made me very happy that MJ had checked the weather prior to planning the weekend. It was the perfect day to be on the west side of Ireland. The harbor there in Galway was beautiful.

Thankfully we were able to flee the city before the parties over the hurling game win broke out, and we were on our way to the Cliffs of Moher.

There are two things I’ll say about the roads out there in western Ireland. First, it’s beautiful out there. The countryside is dotted with castles and various ruined towers, livestock grazing in fields that have stone walls (I started calling them “cow castles”), several rivers and lakes, and so much green.

The second thing is that the roads are terrifyingly narrow! I’m not sure it’s a trip I’d repeat soon given how queasy I became as we drove through the countryside. The feeling when a giant tour bus came careening down the road partially in our lane is not one I wish to repeat in the near future. Still, we did stop a few times to soak in the views, especially as we drove up through the hills.

It was late in the afternoon by the time we made it to the Cliffs of Moher. It was worth the narrow road drive, the cliffs are gigantic and amazing! We got there before the clouds rolled in, and while I was walking around taking pictures the fog started to take over for a bit. The day transformed from warm and sunny with clear visibility to cool and damp with fogged over cliffs very quickly. The change completely altered the feel for the place. One moment feeling alive and fresh, the next gloomy and somewhat ominous. I’m grateful we were there to experience the cliffs in both moods.

After seeing the one side, we walked up the other side to look out over in the direction of O’Brien’s Tower. We did venture a bit past the confines of the visitor center grounds, but I had no compulsion to jump any other fences. We also generally kept our general wandering to a minimum, with MJ still walking on his bad foot and my ankle acting up from the sprain I got at Newgrange on Friday. We were quite the pair, but at least we matched.

There were several selfies taken, and MJ got this shot of me as the clouds poured between the cliffs, but my favorite picture was the one MJ took with his phone and didn’t show me until we got home. It’s probably my favorite recent picture of us!

We left for the long drive back to Dublin just as the sun was starting to set. If Ireland hadn’t already made me fall in love with her back in 2010, this drive back would have done it. With every mile through the countryside my heartstrings were pulled as I gazed at the rolling hills we were driving through at dusk. Photos don’t do it justice. Ireland clearly didn’t want us to leave.

More photos from Galway and the cliffs here:

We got back to Dublin just before midnight and had one final meal at Zaytoon before parking the rental for the night, which we’d drop off at the airport on our way home in the morning.

The spoiled American in me would miss some of the food and creature comforts if I ever moved away from the United States, but Ireland is on my short list of places I could see myself living. Every time I visit it feels like I’m going home and my heart sinks every time I leave. Alas, it was time ! Caligula was waiting for me, and I’d been away from San Francisco for nearly a month.