• Archives

  • Categories

  • Other profiles

  • wallaceadngromit.net

  • Partimus

  • Xubuntu

  • DC/OS

Grace Hopper Celebration of Women in Computing 2017

Last week I had the pleasure of attending the Grace Hopper Celebration of Women in Computing for the second time. The first was back in 2015 when I spoke during the open source track. This year I was attending to represent Mesosphere at our booth in the career fair, we were a silver sponsor of the event.

I arrived on Tuesday afternoon and met up with my colleagues Amita Ekbote and Susan X. Huynh. I was able to get checked in for the conference, have some lunch together and visit the booth in the expo hall to confirm our delivery of stickers and t-shirts had arrived. Since we work on different teams at Mesosphere, it was really nice to spend the afternoon together talking about our work, the company, and thoughts around diversity initiatives that were inspired by our attendance at the conference.

GHC is a very career-driven event, which is showcased in what is not an expo hall, but a career fair. Companies from around the world pour a lot of money on event sponsorships and booth setups to attract talent to their booths, whether in the form of internships or full time employment. The conference had 18,000 attendees, mostly women, and that’s a big pool to draw from.

This was my first time doing booth duty with this kind of focus, but I thoroughly enjoyed it. As a startup, our booth was modest but we have a great logo and the traffic at the booth was brisk throughout the event. I met a lot of women who are interested in security, which I was very glad to see, since the field of security in technology is quite male-dominated. More applicable to our work, I did meet several women who had experience with distributed systems, some of whom had even worked with Apache Mesos before. There were a few times when I got to go off into the weeds a bit, geeking out with an engineer who came to our booth and was enamored with infrastructure work.

Margaret Sy, me, Shira Oatis, Amita Ekbote and Susan X. Huynh, credit to Shira for the sticker display!

There’s something incredibly special for me about getting to work and geek out with other women. As a woman working in the intersection of infrastructure and open source it’s rare for me to be able to. Throughout my entire career, the vast majority of my colleagues have been men. While I’m very happy with the men I’ve worked with, there’s a different connection I have to other women that I really enjoy when I can find it.

And one afternoon I also spent some time with my droid buddy R2-D2. Thanks to the Disney booth!

Now I wasn’t able to make it to any of the sessions at the conference, but I did watch the keynotes. They showcased an incredible lineup of women in technology. On Wednesday morning at the conference kickoff I really enjoyed hearing the funny anecdotes from Melinda Gates as she walked us through her teenage love for her Apple III (“this isn’t Bill’s favorite story”) and the failure of Microsoft Bob, a project that she was a project manager for. She concluded her talk by imploring us and the industry to examine and appreciate non-standard entrances to computing. I loved computers as a teen, so did Melinda, we could geek out about old computers we had all night, but we’re the ones who succeeded. Not every woman, or person, comes into computing that way, but it is the path we focus on and venerate. We need to embrace different paths to careers in tech and make sure we’re reaching the women who don’t match our stereotypes.

The speaker I was most excited about on Thursday was Debbie Sterling, the Founder and CEO of GoldieBlox. Thanks to the grape vine that is the women in tech community, I knew about and I was one of the early believers in GoldieBlox. I voted in the campaign to get them an ad in the Super Bowl which catapulted their success. Hearing her story of following her passion and incredible resistance and rejection she faced before finally succeeding to get VC and funding through a Kickstarter was inspiring. The whole idea of GoldieBlox was also something I believe in, we need more toys for girls that allow them to create and build, and not be read-only participants in the pink aisle.

The other thing that jumped out at me was how incredibly accomplished every single one of the keynote speakers was. At most conferences I attend the keynote speaker may have written a book or founded an organization, but every woman up on the keynote stage as the days progressed had several noteworthy accomplishments under her belt, and I’d never heard of many of these women. Between the keynote speakers and the sold out audience, anyone attending this conference must walk away feeling like we have an incredible talent pool of women in tech in this country.

Theme-wise, there were a couple that jumped out of me. The first was the diversity of their speaker line-up, and how many of them came from less affluent communities. There were women from Ghana, Ecuador, Rwanda, Mexico and China, many of whom immigrated to the US and all of whom have overcome discouragement and adversity to push ahead and achieve incredible success. I’m sure I wasn’t the only one in the audience who was then doing a great deal of self-reflection. We struggle with and are exhausted by microaggressions and harassment throughout our career, but these women have had to overcome so much more and still believe that it was worth it. Improvements of the “first world problem” type definitely still need to be made since they do still take away from the time we have to work and succeed, and still cause women to leave in significantly higher rates than men, but perspective I gained from the experiences of these women was important for me. Keep going, it’s worth it to be players in industry that builds our future.

The second theme was one in the direction of Artificial Intelligence (AI) and Machine Learning (ML). Both of these fields are already having an impact on our lives today, and will to a much greater extent in the future. From Dr. Fei-Fei Li we got clear examples of how “machine values are human values” and that even the most well-intentioned developer still has unconscious bias that can lead to problems where they have blind spots due to their background, gender, race and more. The AI and ML fields need diversity of engineers, data sets and people who ask real human problems when crafting technological solutions.

Finally, throughout the keynotes there were projects and initiatives that speakers mentioned, the following are a handful that stood out for me:

You can watch the keynotes from each day by following these links, and details about the speakers can be found here (where I got these links from):

I had several great meals with my colleagues, but no conference would be complete without a party, and Women Techmakers put on a great one over at the Epcot World Showcase. From 7 to 9:30 we enjoyed small plates and drinks, a great pro-women DJ, a bunch of activities brought in by Google, and even a Waymo self-driving car! I got to check out the new Pixel 2 and Pixel 2 XL, which was great since I think the Pixel 2 XL will be my next phone and getting to play with one was really nice. I also stood in line with my colleague Margaret to get “selfie stickers” printed by Google Allo, which took a picture of you and then generated a cartoon image from it (mine are here).

At 9:30 the park closed to the general public and we had 90 minutes to explore it on our own! The countries in the World Showcase were all closed, and no wine and food festival for us, but we did get to go on the iconic Spaceship Earth ride and Soarin’: Around the World before the evening wound down. Unfortunately my sprained ankle wasn’t a huge fan of this late night, but I’m glad I pushed myself a little so I could take advantage of the evening.

The conference concluded on Friday with a couple final shifts at our booth, during which my boss, and company co-founder, Ben Hindman joined us at the conference and the booth for the day to meet with folks who came by. The four of us remaining then took the opportunity to swing by the eBay booth to pose in front of a great quote from Grace Hopper, “The most dangerous phrase in the language is, ‘We’ve always done it this way.'”

Thanks to Shira for sharing this photo (source)

I decided to depart before the afternoon keynotes in order to avoid rush hour traffic and the rush on cars for hire, but I did watch them later. I was really amused by Ayanna Howard’s keynote where she shared that her studies show that people trust robots, even when they’re not working properly, so there needs to be caution and thoughtfulness involved as we program them. I also had a particular fondness for Maureen Fan’s keynote where she talked about her company, Baobab Studios, and her livelong love for animation. I share this love, and for the same reasons of hope, dreams and taking you to faraway and imaginary places.

In all, I enjoyed the conference and all the people I had the opportunity to meet. It’s nice to be reminded now and then that there are a lot of women working in tech, even if the percentages aren’t where we want them to be. We’re here, and we belong.

A few more photos from GHC here: https://www.flickr.com/photos/pleia2/albums/72157686015448772

A few days in September

I spent about half of September traveling, but between trips to Dublin and Los Angeles I was able to spend a little time at home in San Francisco to get kitty snuggles, observe the Jewish High Holidays with MJ, speak at a local conference, get my ankle looked at and celebrate my birthday. I’ve been busy.

We spent Tuesday through Friday at home between Dublin and Los Angeles. I was able to catch up with a bunch of work post-travel and went into the office on Wednesday, as I do when I’m in town. I ducked out of my office a little early to head over to the Microsoft office in San Francisco. It was just before MesosCon and I wanted to meet up with my fellow DevOps track lead to make sure we were all set. We enjoyed some coffee and went back to to the office to hack on a few things before evening crept up and I had to make my way home. It was fun, I’m not much of an office person but I do enjoy visiting other people’s offices to work now and then.

The condo has been hot this month. Thankfully we missed the record-breaking 100+ temperatures, but even when it’s 85 degrees outside, our 11th floor unit gets very little breeze and it heats up quickly. A small standing A/C unit helps a little, but it’s mostly for taking the edge off the worst of the heat. The heat got even worse one night when our old toaster oven caught on fire. I quickly learned how much of a pain it is to clean up after a dry chemical fire extinguisher, but no damage was done. We picked up a new toaster oven after I got back from LA, and Caligula was pretty excited because he got to enjoy the toaster oven box until I put it away.

I returned from LA just in time for MJ to head out of town the next day for work, just a few days before Rosh Hashanah. But I did have him back in time to celebrate together on Thursday the 21st. We typically go to the evening services and the daytime services on Rosh Hashanah and this year was no different. The daytime services concluded with a Tashlich at Crissy Field, near the Golden Gate Bridge and it was a beautiful day for it.

I also attempted to make a round challah! It was more square than round, but I also did an experiment to make it with honey and it turned out really nice and moist. So next year I know to still make it with honey, but work harder on actually making it round.

The day after a bunch of standing and walking for Rosh Hashanah, I decided I needed to see a doctor about my ankle. While I was at Newgrange in Dublin I twisted it pretty badly. Walking on it in Los Angeles through tar pits and two conferences didn’t do me any favors. I think standing on it during services was the last straw, it was hurting that day I went to the doctor more than it had when I first injured myself three weeks before. That trip to the doctor led me to get x-rays, and x-rays hinted at a break which sent me to an orthopedist the following Monday. Of course that meant I spent the whole weekend worried about whether I’d broken my foot. Ugh.

Thankfully there was no break, but I did get to discover the open source Aeskulap DICOM Viewer and so I got to see pictures of my bones! I was instructed to rest my ankle, since I had clearly done more damage to it. It kept me working from home all week, except for a local conference on Wednesday where I was giving a talk. Fortunately, the talk was only fifteen minutes long and after spending a bit of time at our booth, I was able to grab a ride back home for some more ice and Aleve.

The talk was a shortened version of my Day 2 Operations talk. Given the 15 minutes I had, I was able to take a more playful spin as I explored four myths around operations and containers:

  1. Containers will solve all your problems! (they solve a lot of problems, but create new ones)
  2. Green fields! (no, you still often have complicated legacy infrastructure)
  3. Everything is already automated! (it’s not, but most things have APIs now to hook into existing tooling, nice!)
  4. No more planning! (easier to deploy, but make time to add in logging, metrics, maintenance plans)

I enjoyed the 15 minute format, it felt a bit rushed, but it also caused me to yank out a lot of slides that were “here’s a list of technologies that do this” and focus more on the overall message. Though I always do feel weird telling ops folks they still need logging and metrics in this world of containers, the message is always well-received as they bring it back to their workplaces to argue for the time to build better a more maintainable system. Slides here.

Thanks to Rachel Dines for taking a photo during my talk! source

I wish I had been able to spend more time at the conference though, it was the Sysdig Camp-Con-World-Fest-Summit and there were a number of generalist container and metrics talks that would have been interesting to see. It was nice to meet up with fellow speakers at a speaker dinner at The Slanted Door in Ferry Building the Monday before the event though, we even got to eat clouds for dessert! Alas, foot-wise I needed to take care of myself.

That night I had to cancel some of my other plans as well, but I did pay for a ticket to see Ellen Pao speak on Resetting Silicon Valley so that made my priority list. In spite of my sore ankle, I’m glad I went. Stories of systematic sexism are never easy to hear, but I really enjoyed hearing about Project Include and their 87 Recommendations for companies looking to increase diversity and inclusiveness. The interview with Pao gave me hope for the future of diversity and inclusiveness, as not only projects like these are popping up, but there’s a much stronger understanding of and push back against the now acknowledged problems of sexism, racism, ageism and more in Silicon Valley.

Friday was my 36th birthday! The chaos of travel lately and the evening of my birthday landing on Yom Kippur made it so my birthday was considerably more low key than it’s been in previous years. We skipped a proposed getaway up to Tahoe in favor of a much-needed weekend at home, and went across the street to Fogo de Chao for a wonderful, filling birthday meal before the sun set and we were off to Kol Nidre services and into a day long fast. My best friend did send me flowers though, which was an incredibly sweet surprise. I also accepted the free dessert offered at Fogo, having been interested in their grilled pineapple dessert for some time but usually too full at the end of my meal to even think about dessert. They decorated accordingly!

Yesterday morning I met up with my friend Laura for breakfast. She’s in town for the huge Oracle conference that’s been under construction at Moscone for several days. One of my colleagues is speaking at the conference, but I’m off to Orlando tonight to help out with the Mesosphere booth at the Grace Hopper Celebration of Women in Computing. We only get to catch up in person a couple times a year, and now that my role is on the developer advocacy side, I have a ton to learn from her experiences doing this professionally for several years. She shared tips about working with a budget, “on call” rotations for covering social media, Slack and other community-driven resources and goal setting. In spite of doing this on a volunteer basis for various communities over the years, there’s a lot to learn once you’re in a position when you’re being paid to help engage a community, even if you’re passionate and excited about your work. I have much to think about after our conversation and I look forward to sharing it with the rest of my team.

Grace Hopper will take up the rest of my week. On Friday after the event concludes I’ll be staying with some friends for the night, one of whom is another developer advocate who I met through the Ubuntu community. No doubt we’ll have lots to talk about and catch up on. I’m flying home Saturday night, and staying in San Francisco for just under two weeks before I’m off again. Next adventure? All Things Open, MesosCon in Prague, various Meetups in Germany and then CubaConf. I’ve got my month cut out for me! I’m also pleased to have booked our trip back to Philadelphia over Thanksgiving and gotten kitty care worked out for that. It’ll be nice to have some family time and get back to the townhouse for a week or so.

Sunday at La Brea Tar Pits and Venice Beach

Our journey on the Coast Starlight last month put us down at Union Station in Los Angeles and we grabbed a rideshare over to our hotel for the night. Our hotel had a robot, but unfortunately it was out of service. Poor robot.

Sunday was our tourist day! MJ had to fly back home to get to work on Monday, and I was changing hotels to attend the Open Source Summit Monday morning. The main thing on my agenda was finally getting to see La Brea tar pits. I’d been down to Los Angeles several times, but it’s always been on business, and since it’s so close to San Francisco I often don’t extend my trip to do much visiting. This was an opportunity to finally see the tar pits, and actually do it together!

First we decided to have a leisurely morning. Enjoy breakfast outside, grab a local microbrew and spend an hour at the hotel pool. We left ourselves a bit of time to shower and pack up before checking out and finally making it out to the tar pits.

We arrived in the late afternoon, which gave us enough time to visit the museum before it closed. The museum is full of ice age fossils, some of the more impressive being the mammoths and sabre-toothed cats.

Formerly know as an American lion, I learned that it’s actually more closely related to the modern jaguar, sabre-toothed cat behind it

I’d also seen the wall of 404 dire wolf skulls in documentaries about the tar pits before, but seeing it in person was impressive.

As the museum hours wound down, we decided to walk around the tar pits themselves for a bit. I think the order we did things in was a mistake. It would have been more enjoyable to see the exhibits with a bit more context than the inside of the museum itself provided. The museum is on the older side and the exhibits themselves hadn’t been refreshed much over the years. They seemed to be more on the side of impressing kids with a thin layer of facts, some of which were out-dated, rather than getting into the history of the tar pits themselves. I did enjoy it because the somewhat campy feel of the museum was appealing to me, but I do like good modern museums too.

Now the signs outside at all the pits themselves filled the gap the inside of the museum left. If I were to do it again, I would have started at the side of the park where the Observation Pit is.

Signs explained that the observation pit was the oldest museum on site, housing a tar pit and still actively giving tours there in association with the museum, though they were over by the time we got there. From there a walk around the tar pits brings you close to a bunch of signs, talking about the history of the pits, how they were created and re-discovered, answering questions.

The outdoor pit area included the iconic Columbian mammoth struggling to free from her tar pit fate while two other mammoths look on, and then a sign next to it debunking the scene as it’s depicted. I thought this was really good. The sculptures were put in decades ago and are now famous along with the site so it would be sad to see them go, but the outdated portrayal is unfortunate.

The walk around the pits also brought us to Project 23. Millions of fossils were found as they were building the new Los Angeles County Museum of Art nearby. Instead of halting development for years, huge chunks of tar pits were removed to be examined later and building continued on the site. The boxes were put in their current place in 2008 and the project has been ongoing ever since.

After our tar pit adventure it was time to find dinner. We headed out to Venice Beach. MJ had been with colleagues before, but it was all new to me. We ended up at L.A. Gastronomy where I had a lovely dinner of yellowfin poke on rice with seaweed, sesame, caviar, daikon and scallions, along with an appetizer of their short rib poutine.

After dinner we still had a bit of time to walk around the pier at the beach nearby to watch the sunset before MJ’s flight. We even saw some lightning off the coast, which is not the most common thing on the coast of California.

With the sunset concluding our evening, it was back to the hotel to pick up our luggage and go our separate ways. It was a short getaway, but it was relaxing and I really enjoyed finally getting to see the tar pits.

The Coast Starlight

I’ve wanted to take a journey on Amtrak’s Coast Starlight for years. Alas, timing was always a challenge. It’s so hard to justify spending 11+ hours on a train when a short flight will suffice, but 2017 has been my year of trains. Over Memorial Day MJ and I took the California Zephyr and Capitol Limited to get across the country and this month I’ll be hopping on the Carolinian with a close friend, and fellow train adventurer, from Philadelphia to Raleigh.

We finally decided to take the Coast Starlight down to Los Angeles a couple weeks ago to get me to the Open Source Summit and MesosCon.

Now I’m going to pause for a moment to talk about my feelings about Amtrak. It’s the only long-haul passenger service in the United States, and I love trains. It makes it my de facto home train so I have a special fondness for it. But Amtrak is expensive, often more so than flying, and the cars are extremely dated. The food even in the dining cars isn’t great, and there is no variation in the menu between routes that I’ve seen thus far. They serve a fair amount of pork, which excludes much of the menu for me and making the menu even smaller. I can’t imagine riding on it as a vegetarian unless you packed your own food. The air conditioning leaves much to be desired and even if you get on a train that advertises having WiFi, your best bet is to assume it’s not working. The WiFi situation often isn’t their fault, coverage is simply spotty through some of their service area, which leads me to another thing out of their control: the United States is big and taking a train that typically must travel below 79 MPH is going to be slow.

Still, it’s my train and I enjoy taking trains. The views, the disconnection, the dull roar of the rails. I can relax, get some reading done or chat with my fellow passengers in the lounge car.

MJ and I departed early on a Saturday morning to begin our trip at an Amtrak bus station near the Embarcadero here in San Francisco. Unfortunately our train was delayed, as I hear often happens with this route, so after the bus dropped us off at Jack London Square station we had a couple of hours to kill while we waited for the train. It did finally arrive though!

In spite of not doing an overnight this time, we sprung for the Roomette so we could have have private space to retreat to during our 11 hours on the train. This class of room got us access to the famous Pacific Parlour Car. The status of the six Pacific Parlour style cars is tracked on pacificparlourcar.com, explaining that five of them are in operation by Amtrak on the Coast Starlight on train numbers 11 southbound and 14 northbound. These are pretty special car in the Amtrak fleet, they’re Hi-Level lounge cars by The Budd Company in 1956 for the Santa Fe Railway, not Amtrak. They’re also shorter than the other cars, which you notice a little when walking between the cars but it’s definitely noticeable from the outside of the train. As for the features of the car itself, this page has a photo tour that takes you through the bar area, seating area, dining area and downstairs to the movie theater.

The Parlour Car

In the afternoon the parlour car also hosted a wine tasting, but while it was fun to move around a bit and have a few sips of wine, the wine snob in me was not terribly impressed with the selection. I did manage to have a Stone IPA during dinner though, which somewhat to make up for the lackluster showing from the wines.

The downstairs movie theater was pretty cool, unfortunately they weren’t showing any movies. When I asked about the movie theater one of the staff members said the royalties were too expensive so they currently couldn’t show any. I’ve also read on the forums that they’ve struggled with the technology aspect of it, switching to DVDs when satellite feeds wouldn’t suffice. I couldn’t stop thinking about this for the rest of the trip. What an opportunity for someone interested in developing a Creative Commons program! Old movies! Modern, CC-licensed cartoons! So many great options! Maybe some folks would be disappointed that it wasn’t known blockbusters, but it’s quality content and certainly better than nothing. Develop a reliable SSD-based box to stick in there to play the content on the TV and then you get to avoid DVD and satellite issues. I actually now want to be the crazy tech lady who runs the movies on the Coast Starlight.

All that said, I probably wouldn’t have watched a movie during the trip. Hah! As my first journey on the Coast Starlight I was eager to see the scenery down the whole route.

Down through the bay area we had the predicable salt marshes, a group of hikers here and there. As we went further south we passed through a lot of farm land. Here and there the backdrop of mountains made for some views worth seeing.

I also managed to get a panoramic photo as the train navigated the horseshoe curve at Cuesta Pass. Zoom in on the photo, you’ll see both the front and end of the train! Our car was near the middle of the train, and it takes a few seconds to take the panorama, making the following photo possible.

And then just south of San Luis Obispo we got to the amazing part of the trip. The train heads west and runs along the coast line. We enjoyed dinner as we looked out over the cliffs and beaches.

No joke, RVs, surfers and palm trees ruled down here.

The sun set as the clock ticked past 8PM and the rest of the trip was taken in darkness as we passed through Oxnard and Burbank. It was interesting to learn that the Burbank airport is attached to the Amtrak station, which is a rare thing here in the US, but obviously makes a lot of sense. We got into Union Station in Los Angeles just after 9:30PM on Saturday evening. Onward to our Sunday of adventures in Los Angeles!

Sub-optimal air-conditioning aside, I really enjoyed the trip down and I’m glad we finally took the time to do it. More photos from the trip down here: https://www.flickr.com/photos/pleia2/albums/72157685483861352

Galway and the Cliffs of Moher

Our last day in beautiful Ireland this year was spent on the west coast. MJ had been out there, but it was the first time for me. He’d also driven on the left side of the road before, so when it came to renting a car, he was the driver. That morning we left the hotel early to pick up or lovely BMW rental car and make our way out to Galway!

The drive took about two and a half hours, we found parking and quickly discovered we were in the midst of a city that was extremely excited about the hurling championship game that Galway was playing in. I knew nothing about the game when we walked into The Quays for lunch. I still know very little about it, but we had a lovely lunch with one of MJ’s colleagues and his wife who also happened to be on that side of the country for the weekend. Plus, I got my beloved fresh north Atlantic oysters.

After lunch the sun came out and we walked around town a bit, walking right past Lynch’s Castle (a medieval “town house”) without realizing it as it’s a bank these days. We then walked past a couple of old churches before making our way down to the Spanish Arch and then back to the car. The morning had been rainy in Dublin, but the stunningly blue sky that met us after we saw the arch made me very happy that MJ had checked the weather prior to planning the weekend. It was the perfect day to be on the west side of Ireland. The harbor there in Galway was beautiful.

Thankfully we were able to flee the city before the parties over the hurling game win broke out, and we were on our way to the Cliffs of Moher.

There are two things I’ll say about the roads out there in western Ireland. First, it’s beautiful out there. The countryside is dotted with castles and various ruined towers, livestock grazing in fields that have stone walls (I started calling them “cow castles”), several rivers and lakes, and so much green.

The second thing is that the roads are terrifyingly narrow! I’m not sure it’s a trip I’d repeat soon given how queasy I became as we drove through the countryside. The feeling when a giant tour bus came careening down the road partially in our lane is not one I wish to repeat in the near future. Still, we did stop a few times to soak in the views, especially as we drove up through the hills.

It was late in the afternoon by the time we made it to the Cliffs of Moher. It was worth the narrow road drive, the cliffs are gigantic and amazing! We got there before the clouds rolled in, and while I was walking around taking pictures the fog started to take over for a bit. The day transformed from warm and sunny with clear visibility to cool and damp with fogged over cliffs very quickly. The change completely altered the feel for the place. One moment feeling alive and fresh, the next gloomy and somewhat ominous. I’m grateful we were there to experience the cliffs in both moods.

After seeing the one side, we walked up the other side to look out over in the direction of O’Brien’s Tower. We did venture a bit past the confines of the visitor center grounds, but I had no compulsion to jump any other fences. We also generally kept our general wandering to a minimum, with MJ still walking on his bad foot and my ankle acting up from the sprain I got at Newgrange on Friday. We were quite the pair, but at least we matched.

There were several selfies taken, and MJ got this shot of me as the clouds poured between the cliffs, but my favorite picture was the one MJ took with his phone and didn’t show me until we got home. It’s probably my favorite recent picture of us!

We left for the long drive back to Dublin just as the sun was starting to set. If Ireland hadn’t already made me fall in love with her back in 2010, this drive back would have done it. With every mile through the countryside my heartstrings were pulled as I gazed at the rolling hills we were driving through at dusk. Photos don’t do it justice. Ireland clearly didn’t want us to leave.

More photos from Galway and the cliffs here: https://www.flickr.com/photos/pleia2/albums/72157689480206285/

We got back to Dublin just before midnight and had one final meal at Zaytoon before parking the rental for the night, which we’d drop off at the airport on our way home in the morning.

The spoiled American in me would miss some of the food and creature comforts if I ever moved away from the United States, but Ireland is on my short list of places I could see myself living. Every time I visit it feels like I’m going home and my heart sinks every time I leave. Alas, it was time ! Caligula was waiting for me, and I’d been away from San Francisco for nearly a month.

A Castle and the National Transport Museum in Howth

I spent Friday during my stay in Dublin visiting Newgrange and the Hill of Tara on my own, with twenty of my newest tour group friends. Come Saturday MJ and I had the weekend together! We planned for one day to include a trip to Galway and the Cliffs of Moher, and the other to spend local to the Dublin area. After consulting weather reports, Saturday became the day we stayed near Dublin. However, we’d both already done all the major sights in Dublin itself. Instead we decided to hop on a DART train and head out to Howth. What’s in Howth aside from some nice views? A privately owned castle you can wander around, and the National Transport Museum!

We had a leisurely morning that allowed us to get to Howth in time for lunch. We ended up with a great lunch outside at the restaurant attached to the Howth train station, The Bloody Stream. Gotta love Viking-inspired naming! We then walked over to the transport museum.

The transport museum is tucked behind Howth Castle. It’s home to dozens of vintage vehicles housed in a pair of warehouses, with a strong preference for buses used throughout the 20th century. There were also work vehicles, a firetruck going back more than a hundred years, and a few things that ran on rails. I’m a rail fan, so some quick research on their website ahead of time clued me in to the handful of vehicles that would be of particular interest to me.

The first was their tram No. 253. With the exception of tourist buses and Amtrak, double-decker vehicles are pretty rare in the United States. I’ve never seen a double-decker tram (streetcar) running before, as none of the lines I’ve visited were double-decker capable. However, they’re incredibly common in Europe. From vintage trams to modern buses, the double-decker style in the British Isles seems to be the norm rather than the exception. This means that tram 253 and the other two I visited were all double-deckers. What a treat!

Tram 253

The next tram I was visiting was right nearby, decked out in varnished grained teak was tram No. 9. I was more excited about 253, but this one was just stunning. It had been beautifully restored and shined from it’s corner of the museum.

Tram 9

After visiting these trams, I caught sight of No. 224, one of the most popular vehicles in their collection. At first glance it’s a tram, but closer inspection reveals the truth: it has tires, and “false tramway truck frames were fabricated, giving the tram a highly authentic appearance.” Not only that, it’s a former trailer that’s been so extensively rebuilt that it is now serving as a replica of the original Dublin No. 224, a former trailer, which records show met it’s demise around 1923. The full story is on their website, a fascinating read for people interested in the sorted history which also includes the death of a passenger in 1898 that led to a change in how trailers were used in Dublin at the end of the 19th century.

Replica Tram 224

I do happen to like fire trucks as well, so photos of them and various other vehicles in the transport museum made their way into my album for the museum: https://www.flickr.com/photos/pleia2/albums/72157687000431984

Once we finished walking around the Transport museum we made our way back to Howth Castle. Since it’s a private residence, tours are only done on a booked schedule, and only certain days of the week. Still, the owners allow people to walk around parts of the grounds, so it was nice to walk around a bit and take some photos. See, as the stereotypical American, I love castles.

More photos of the castle and other adventures around Dublin throughout the week are in my Dublin and Howth album: https://www.flickr.com/photos/pleia2/albums/72157689479792585

Newgrange and the Hill of Tara

At the beginning of September MJ and I spent some time in Ireland. With MJ wrapping up his work week in the Dublin office, I spent the Friday on a tour bus visiting some sights. He had already been to the Neolithic passage tomb at Newgrange, so that was number one on my list. The day long tour I booked also included a stop at the Hill of Tara (Cnoc na Teamhrach).

The weather was beautiful. The tour began with a quick drive through the south of Howth, just outside of Dublin, where we saw some beautiful views across Dublin harbor. There was then a half hour stop in the main harbor area for some fresh air.

From there we were off to Newgrange. The tour bus got us there just before noon and we all went to the visitor center. At the visitor center we browsed for a few minutes as we all waited for the tram which would take us up to the site itself.

Archaeologists estimate is that it was built around 3200 BC, predating both Stonehenge and the great pyramids in Egypt.

When it was rediscovered by archaeologists in modern times, it was overgrown and a lot of the stones had been taken away from the site to build nearby structures. Several excavations happened throughout the 20th century, and the controversial reconstruction we see today, which includes the retaining wall, was worked on from the 1960s through the 1970s by Professor Michael J. O’Kelly. It is a little sad to know that the exterior is a restoration, however researched and informed, but it was still an impressive place to visit.

The tour took us past the “most photographed stone in Ireland” – the entrance stone which is carved with swirls of meaning that has been lost to time. I took a picture of it too. As advertised, past the stone is the entrance to the mound, where we were lead down a low and narrow passageway which reaches about one third of the way into of the mound. At the end of the passage there’s a trio of alter-like spaces which are believed to be ceremonial, as well as being where period human remains were found. Lending further credence to the ceremonial speculations, on the winter solstice the entrance is aligned with the rising sun, an experience that people enter a lottery annually to get a chance at participating in it. They do a mock sunrise with artificial lights during the tour, to give visitors a hint of what the experience is like.

After visiting the interior, we had another half hour or so before the tour bus returned to pick us up. I took the time to walk around the mound to take more photos, and sprain my ankle. Only one of these things was intentional, but they were related, fiddling with my camera while walking on uneven paths is clearly too dangerous for me. A sprain is not the souvenir I planned on returning home with! Unfortunately I then continued to walk on it, not just that day, but over the whole weekend, at a conference two weeks later, and I’m just now trying to properly attend to letting it heal.

Once back to the visitor center we had time to get lunch in the cafe, stop in the shop and take a quick browse around the on site museum. The pace of guided tours is often unfortunate for me, I could have spent more time there, but we had another stop on our itinerary, the Hill of Tara!

I’ll be honest, in spite of the historical significance of the Hill of Tara, there’s not much to see there. The story goes that it was the seat of the High Kings of Ireland for centuries. As the tour guide brought us through the complex he explained some of the known roads and buildings that evidence was found for. Another passage tomb is also located there, known as the Mound of the Hostages. But the site is mostly a series of rolling hills, not optimal on my ankle, but I was still confident in my ability to walk it off.

Mound of the Hostages

The most impressive spot is the the Stone of Destiny (Lia Fáil), said to be where the High Kings were crowned. The guide explained the legend of the stone, sharing that the true king, having met a series of challenges, would touch the stone during the ceremony and the stone would scream, being heard throughout Ireland. I touched the stone, it didn’t scream.

Touching The Stone of Destiny (Lia Fáil) atop the Hill of Tara

At the conclusion of the tour the guide was kind enough to drop me off at MJ’s office, as he was heading in that direction to drop off a couple anyway. I was able to get a quick tour before we headed back downtown and had a wonderful dinner at The Bank Bar and Restaurant right near our hotel. There we had some drinks and dined with the bust of James Connolly.

With Friday winding down, the weekend awaited us!

More photos from my day here: https://www.flickr.com/photos/pleia2/albums/72157688735097416

Open Source Summit NA 2017

I recently attended my first Open Source Summit, formerly LinuxCon. It’s strange that having worked in the space of Linux for so long that I kept missing these Linux Foundation events, I think the only event of theirs I’ve been to was the Linux Collaboration Summit in 2010. I always had conflicts, then differing priorities with regard to events, especially since I’ve rarely been paid to actually do work directly on Linux, it’s always been a hobby or base for other projects and infrastructures I’ve worked on. The co-location with MesosCon and shift to general open source conference changed the game for me, so I could finally attend and speak at this North America edition in Los Angeles!

I was thrilled to have a talk accepted at the Open Community Conference run by Jono Bacon. My talk was on Building Open Source Project Infrastructures where I explored the current state of proprietary infrastructures used to assist open source project and asked the audience to consider the benefits of self-hosted and open source solutions for their projects. The OpenStack project and others listed on opensourceinfra.org have open sourced various parts of their project infrastructure to allow for outside contributions and even assistance from the community in hosting. Both the Xubuntu and DC/OS projects have websites that are hosted in revision control. Several projects are using open source tooling for their continuous integration work, and publishing your build system helps contributors replicate releases when adding their own patches and improvements.

I then walked the audience through steps to opening up their infrastructures more, including finding talent (systems administrators do exist in open source projects!), determining what hosting is required, and seeking funding to support anything your project may end up needing to pay for. The talk concluded with a closer look at OpenStack and Xubuntu to walk through the different ways these projects are maintained and the various pieces of the infrastructure we put in the hands of community members. Slides here (PDF).

Thanks to Shilla Saebi for taking a photo during my talk! (source)

On Tuesday I spoke in the ContainerCon track on Advanced Continuous Delivery Strategies for Containerized Applications Using DC/OS. This was actually a talk submitted by a colleague who wasn’t able to participate, so he handed it off to me. I was able to take some slides from other talks, but I always have to do a fair amount of rewriting to make a presentation my own. In this talk I walked through modern continuous deployment pipelines and stressed two things in a containerized infrastructure: running everything in containers and organizing workloads efficiently. DC/OS does this by running both services (Jenkins, GitLab, etc.) inside of containers and software actually being tested. Jenkins has a really nice plugin for Mesos which allows it to farm workloads out to the Mesos cluster automatically, negating the need for something like Nodepool that we used in OpenStack land. The use of containers lends to the second bit, being able to run multiple, isolated workloads across the same hardware instead of using discrete, dedicated servers for various services. Additionally, by using something like DC/OS you can give developers the flexibility they want to use their own tooling without the additional infrastructure administrative needs that exist on for each team running their own hardware/VMs themselves. Everyone gets a chunk of the Mesos cluster and they install what they want, only having to worry about maintaining versions of the applications themselves.

It was after this talk that I learned something fascinating about my references to container. My colleague Jörg mentioned to me that a lot of folks he interacted with automatically assumed “container” to mean “container image” rather than the concept of a running any generic container in the operational sense. I was puzzled by this until I went to a talk later that day by another one of my colleagues, Jie Yu who spoke on Containerization in Mesos, Embracing the Standards. It dawned on me that the communities Jörg tended to work in were developer-focused and mine were operations-focused. It makes sense that we’d see those views in our work. It did cause me to be more cautious about my definitions though, the assumptions I made about what people mean when they say “container” and would indeed be a bit confusing if the assumption is that I’m talking about an image.

Terminology revelations aside, Jie’s talk was an interesting one. He gave some introductory details about how Mesos works, but then dove into the history of containerization in Mesos. This was a history I had mostly pieced together over the past 8 months as I learned about the Mesos containerizer and how Docker fits into the Mesos world, but it was great to see it presented formally, in chronological order so I could more deeply understand the progression and design decisions throughout. He then got to the heart of his talk, where he linked many of the changes being made to improvements users were asking for, as well as the importance of standards in the container space, specifically saying we needed:

  • Stable interfaces
  • Backward compatibility
  • Multiple implementations
  • Vender neutrality
  • Interoperability

He specifically called out efforts by the Open Container Initiative, the Container Network Interface (CNI) and the Container Storage Interface (CSI).

Moving on from my own focus at the conference, I enjoyed the keynotes I saw on Monday and Tuesday (most of the videos can be found here). Though I wasn’t nearly as star-struck as some in the audience, it was particularly interesting to hear from Joseph Gordon-Levitt about hitRECord where they’ve built a community very similar to online open source communities, but for content creation. I believe in the principles of open source, but it’s still inspiring to see that the key concepts of online collaboration are transferable to things beyond software. He also dove into monetary compensation for artists (and developers), which is something we’ve spent many years shying away from as a community. Many of us are paid for our work in open source today, but as someone who came up as an unpaid community member for most of my career, honest, open, discussions about it are not as frequent as I’d like. Video here.

I also really appreciated the keynote from Dan Lyons, author of Disrupted: My Misadventure in the Start-Up Bubble. The talk began by poking fun at the culture of startups, but a third of the way in took a turn for the dark about the dangers of startups and what the fun is hiding. Workers are increasingly being exploited in and around our industry, we need to see this turned around. Video here

During the summit I had the pleasure of meeting Ed Warnicke who gave a lightning talk on Tuesday afternoon about Bringing Network and DevOps People Together. He’s a Principal Engineer at Cisco and was raising awareness around the disconnect between networking and systems folks. This talk hit home for me, literally. When we got married, my husband was working as a network engineer and I was working as a systems engineer. Whenever we talked about networking-related topics, we’d talk past each other. This became even worse when I was working on OpenStack, and then my OpenStack book. I had a networking chapter that I wrote with a fellow systems engineer in the OpenStack community and as soon as I showed it to network engineers, they were confused. I probably spent more time on this chapter than any other because I had systems and networking reviewers pushing back and forth at terminology and ways I explained things and more fundamentally how OpenStack was doing certain things.

Coming back to the talk, Ed stressed that with the future we’re heading into with microservices, we can’t afford to talk past each other and the DevOps world shouldn’t be ignoring the expertise of the networking community that’s been solving problems for decades. His own call to action was for all of us to talk to someone from the other discipline. Done and done! Video here.

There were bunch of other talks I enjoyed as I moved between the Open Community and Containers tracks. It was a little disappointing on Wednesday when my attention was pulled over to the MesosCon Hackathon, but I did make it upstairs to see the puppies.

The event was also great for meeting up with people I hadn’t seen in a while and meeting new people. I enjoyed a women of open source lunch on Monday where we heard an intro blurb from every woman in the room (there were dozens!) and I just happened to sit down next to someone I know online via following each other on Twitter.

First two photos: source, source, Mesosphere Developer Advocates, LinuxChix LA!

…and this collage only captures a fraction of the people I managed to chat with. Great conference.

More photos from the Open Source Summit here: https://www.flickr.com/photos/pleia2/albums/72157687000663014

MesosCon NA 2017

I recently attended my first MesosCon, the North American edition hosted in Los Angeles. There are three such events per year, the other two held in Asia and Europe. These events bring together various companies and other organizations working with and on Apache Mesos so I was really eager to finally meet some of the folks I’ve only interacted with online.

The event began on Wednesday with a Mesos community Hackathon and a customer-focused DC/OS Day run by Mesosphere. Given my role and background, I joined the Hackathon where my colleague Jörg Schad kicked off the event by encouraging attendees to collect ideas for projects they wanted to tackle throughout the day. Topics ranged from documentation improvements, creation of a Kubernetes-related demo for SMACK, work on frameworks and further work on an autoscaler for several cloud platforms where DC/OS runs. Plus, there were mini cupcakes!

The main event for MesosCon began Thursday morning with an introductory keynote from Ben Hindman, Co-Creator of Apache Mesos (and my boss), where he covered some of the latest features in Mesos over the past year that he’s most proud of. These included the introduction of nested container support, adherence to standards through the Open Container Initiative project and beyond, and general expansion in usage and community. Mesosphere Co-Founder Tobi Knaup also got on stage, joined by Tim Hockin of Google for a demonstration of the new beta availability of Kubernetes on DC/OS. One of the really distinctive things about this implementation is that it’s running pure upstream Kubernetes, among other things, this allows for migration of a pure Kubernetes environment (on Mesos or not) as well as known functionality with the tooling written today for pure upstream Kubernetes.

Ben Hindman sharing new and noteworthy improvements to Mesos

These talks were broken up by a pair of panels, the first talking about Mesos in the Enterprise with Michael Aguiling at JPMC, Larry Rau (Verizon), Cathy Daw (Mesosphere), Stefan Bauer and Hubert Fisher (Audi), and Josh Bernstein from {code} as a moderator. At the enterprise level, a lot of interest was around distribution of resources across cloud vendors and on-prem, the introduction of open source software in companies unaccustomed to it, general scaling needs required by large companies and a standards-driven future so they can continue to integrate container technologies into their infrastructure.

The final keynote panel of the morning had Ben Hindman sitting down with Neha Narkhede (co-founder and CTO of Confluent) and Jonathan Ellis (co-founder and CTO of Datastax) for a fascinating discussion about the role of Apache Cassandra and Apache Kafka in the fast-data driven world powered increasingly by the SMACK (Spark, Mesos, Akka, Cassandra and Kafka) stack. The main message in the panel was how fast data is essential in the interaction with customers today, and their Internet of Things gadgets that create an expectation of immediate response. This panel was also where I heard a theme that would be repeated as the conference progressed: people move to microservices incrementally. There are a few ways of moving an environment which is microservices driven, but whether they convert old systems slowly or adopt a policy that all new projects must be built with them, it’s incredibly common across the industry. As a developer advocate these days, it was also nice to hear from Neha that the open source nature of the components are empowering developers, which makes my role in getting out there to talk to developers and operations folks who are on the ground with the technology particularly important.

The keynotes concluded and we dispersed into three tracks, on SMACK, Ops, and DevOps, and a new addition to the conference called MesosCon University where attendees could attend an 80 minute hands-on session on making applications production-ready, securing Mesos clusters, or building stateful services.

I was the track lead for the DevOps track along with Julien Stroheker from Microsoft. Leading up to the conference Julien and I reached out to the speakers to get slide decks and answer any questions, then the two of us met up to chat about the track in San Francisco the week before. The day of the track everything went smoothly, with Mark Galpin of JFrog getting us started by talking about the development and deployment of three key projects: Artifactory, Bintray and Xray. They learned quickly that the application and configuration layers had to be configured separately and that it was best to plan for enterprise and high-availability early on, rather than trying to build it in later. He also discussed importance of good startup scripts and package management in a microservices world, since you don’t simply use an RPM anymore to install things… but you do still need RPMs because not every customer will want to adopt a microservices environment just yet! Standardization was also important, customers wanted the ability to hook into their existing storage and network backends, and often had unusual requirements (for example, “my nodes have no outbound internet access”). He concluded by stressing that everyone in the DevOps organization should be involved from the beginning when working to develop the products and how they’re being deployed so that they consider every aspect of how the tooling will be used and deployed.

The next talk was by Aaron Baer from Athena Health who brought us deep into the world of healthcare information services where monoliths rule and shared his experiences working to bring new technologies build on microservices into the fold. Like I heard earlier in they keynote panel, he’s been working on a path to incrementally move from monolithic and relatively inflexible services over to microservices. In addition to building new infrastructure where they can begin using containers, they’ve also been using their legacy database as a canonical resource and storage, and doing data processing and delivery through faster, more modern data storage and processing tools. He’s also a proponent of code as infrastructure (me too!) so it was great to hear him talk about how essential API-driven infrastructure engineering and automation is to getting our infrastructures to the next level.

Aaron Baer on the progression of monolith to microservice

After lunch the track continued with Chris Mays and Micah Noland from HERE Technologies sharing details about the Deployment API they built for DC/OS. They wanted a simplified version of the core DC/OS API which was deployment-centric and needed an open source option that would improve upon Marathon-LB. The API they’ve developed has just recently been open sourced and can be found here: https://github.com/heremaps/deployment-api. During the talk Micah provided an in depth tour of how straight-forward their YAML configurations were for a sample deployment of a Jenkins pipeline. The API theme continued in the next talk by Marco Palladino of Mashape, the makers of Kong. Kong is an open source gateway for APIs which allows you more control over access, including security, authentication and rate limiting. He stressed how important this whole ecosystem of support is for microservices, and his talk also touched upon the steady migration needed by most organizations from their old stack to the new, which also meant keeping a lot of pieces like their legacy authentication system intact but that adding too many layers can leave to unforgivable latency.

Lively booth area break before the final two sessions of the day

The day concluded with talks from Will Gorman of Cerner on Spinnaker and Imran Shaikh of YP on hybrid clouds. I enjoyed the entire DevOps track, but these two talks really went to the heart of what I’ve focused on infrastructure-wise in the past five years. Give my CI/CD background, I’ve known about Spinnaker for some time, but never made diving into it a priority. This talk inspired me. As you may have guessed, Spinnaker is a continuous delivery platform and Will began his talk by covering the basics about it before getting into integration with DC/OS, including the ability to hook into Metronome for jobs. He also touched upon controls Spinnaker has for the deployment phase to complement Marathon’s own attempts to deploy and restart processes. I’ll definitely be digging into it on my own very soon. Imran’s talk on the importance of being responsible for your own infrastructure was very satisfying for me. He led his talk by talking about how moving to the cloud doesn’t absolve you of the responsibility of making sure you have a good disaster recovery plan, reminding the audience that your business relies upon a good failover plan, and your cloud provider has different goals. Throughout his talk his talking points revolved around vendor independence, distributing risk and the importance of your processes being portable to other solutions.

Will Gorman showing how to execute a Metronome job as a step in a Spinnaker pipeline

Thursday evening had another MesosCon first, Town Hall sessions. A big hit with attendees, these were casual evening gatherings on various topics (Mesos, DC/OS and Marathon/Chronos). They allowed community members to gather and talk about whatever they wanted, swap war stories, share tips, anything! Sadly I had to miss this, participation in the Open Source Summit earlier in the week and with a panel to prep for, I was running low on energy. I’m looking forward to attending the ones we host at MesosCon EU coming up at the end of October.

Friday morning came quickly, and I’ll let you in on a little secret: I had breakfast with my Future of Cluster Management panelists. The panel was made up of Sharma Podila (Netflix), Sam Eaton (Yelp), Ian Downes (Twitter) and Zhitao Li (Uber). As the panel moderator I had worked with them on questions and spoke with them via video conference, I hadn’t met any of them in person. A casual, leisurely hotel breakfast was a great way to break the ice and answer any lingering questions folks had, including myself (“none of you are going to surprise me with the answer you give about the importance of open source, right?”). I highly recommend such a gathering for other moderators if it’s possible, I’ve been a panelist a few times and on stage ice-breaking is not always the best way to go.

Jörg was the MC for day two of keynotes, and after a community-focused introduction to this second day (the first day was more enterprise-focused), he brought us on stage. Talking about the future is always fun, and this panel was no exception. After introductions, we covered the importance of open source in our ecosystem, something that is near and dear to my heart. Theme-wise we talked about the move from perfecting deployments on containerized systems to a focus on maintenance, and the tools they’ve had to built outside of what Mesos provides to manage their clusters. The question of how they handle mixed workloads (stateless vs. stateful, batch vs. long-lived) and what improvements could be made here. The conclusion centered around the future of artificial intelligence of clusters, which led us to learn that some of them are already using machine learning to teach their clusters about load and behavior, which was later explored in a track talk from folks at Twitter who are doing automated performance tuning using Bayesian optimization. The last question I had for the panelists was what they wanted to see in 5-10 years, and their answer was identical: they don’t want to care about the underlying tech, they just want their workloads to run.

Keynote panel: Future of Cluster Management, thanks to Julien Stroheker for the photo (source)

The morning keynotes continued with a talk from folks at the National Geospatial-Intelligence Agency that spoke to how they’re using Mesos at scale inside of their organization. Keynotes concluded with one from Ross Gardler from Microsoft who explored the question of where we were in the hype/actual production cycle for microservices and containers. He expressed that there’s still a lot of diversity in the field, and that with so many early deployments there’s no clear “winner” in the space of containerization. Today, he shares, the focus should be sharing best practices and methodologies across platforms so that all of the next iterations of our various containerization platforms can be that much better.

Just like Thursday, after keynotes we shuffled off into various tracks, with Friday featuring tracks on Mesos Internals, Ops, SMACK and the one I took over as track lead for, Mesos Frameworks. My focus in my role at Mesosphere had largely been on higher level operations, so it was a very interesting track for me to sit in on for the day. There’s a lot of really interesting work being done in the Mesos ecosytem that I’d love for my team to be a part of drawing more attention to. This day was great for that.

The first talk was on the juggling that happens around optimizations, service guarantees and how those trade-offs manifest by Sharma Podila of Netflix, who I’d just met earlier in the keynote panel. He began by asking some questions about your cluster and explaining the heterogeneous nature of the hardware they have at Netflix, you never know which generation hardware your workload is going to land on. He then dove into the challenge of scaling down, showing off the open source Fenzo, “a scheduler Java library for Apache Mesos frameworks that supports plugins for scheduling optimizations and facilitates cluster autoscaling.” His talk was followed by one from Wil Yegelwel for TwoSigma who joined us to talk about simulation testing. In spite of my deep interest in CI/CD, simulation testing is something that’s largely escaped my radar. Thankfully he gave an introduction to the concept and then shared how they used the scheduler they built at Two Sigma, Cook, to intelligently handle scheduling of workloads. The intersection of an intelligent scheduler and a policy of simulation testing meant they could run tests that changed configurations and see how they could optimize things as much as possible, and he explained that this has sometimes resulting in some surprising, non-intuitive discoveries that boosted performance. He also credited simulation testing with helping engineers become more familiar and comfortable with the infrastructures they’re running, which are growing increasingly complicated and difficult to fully understand.

After lunch we dove right back in with Joshua Cohen and Ramki Ramakrishna from Twitter who came to speak on the aforementioned Bayesian optimization talk. It’s well known that Twitter’s infrastructure is built on a lot of microservices. At that scale, they explained, manual performance tuning of hundreds of JVM options doesn’t scale, is error-prone, time-consuming and honestly leads to goal-driven engineers copying existing configs to get going, without fully understanding why the optimizations exist in specific places. They went on to stress that even if you do manage to get everything running well, it’s effectively undone as soon as you change or upgrade any component. Instead they shipped the problem off to an internal machine-learning driven technique that uses Bayesian inference to optimize performance. Bayesian became popular in my own operations world for spam analysis, but the methodologies as I understand them make a lot of sense here as well. Unfortunately they haven’t open sourced the Bayesian Optimization Auto-Tuning (BOAT) tooling, but knowing that it’s been done and is successful for them is a good start for others looking to do similar machine-learning inspired performance tuning.

Next up was Tomek Janiszewski from Allegro who gave us 8 tips for Marathon Performance, summing them up:

  1. Monitor — enable metrics
  2. Tune JVM
  3. Optimize Zookeeper
  4. Update 1.3.13
  5. Do not use the event bus
  6. Use a custom executor
  7. Prefer batching
  8. Shard your Marathon

Marathon is a popular project in the Mesos ecosystem, so the room was quite full for this talk and the questions about it went well into the coffee break we had between sessions!

After the coffee break we were in the home stretch, only two more talks to go before MesosCon concluded! First up in this final pair was Dragos Dascalita Haut and Tyson Norris from Adobe Systems who where sharing details about their use of OpenWhisk as a Mesos framework. They quickly covered OpenWhisk, an event-driven serverless platform for deploying code. Their lively presentation was full of demos and they shared their Akka-based mesos-actor project before pulling in the OpenWhisk bits that brought together all the pieces of the work they were doing.

The final talk brought a pair of engineers from Verizon Labs, Tim Hansen and Aaron Wood, to share their work on fault-tolerant frameworks without Docker. They share some of my own opinions about the over-use of Docker where something slimmer would suffice, so I knew going in that I’d enjoy this talk. The talk explained why using Docker and the entire Docker daemon was often overkill for their work, specifically Tim saying that there was “no real need for an extra daemon and client when Mesos can containerize tasks.” So they instead directed the audience toward use of the Mesos containerizer and the CNCF’s Container Network Interface (CNI). To this end they developed Hydrogen, “High performance Mesos framework based on the v1 streaming API” and their own Mesos SDK, a general purpose Golang library for writing mesos frameworks.

As always, as tiring as the week was it was still sad to see it conclude. It was great to meet up with so many people, huge thanks to everyone who take time to let me pick their brain, or simply sat down for a quick chat. I learned a lot while there and had a wonderful time.

Clockwise from top left: With Yang Lei (IBM), Dave Lester (Apple), Mesosphere community team (minus Jörg, plus Ravi!), and Aaron Baer

More photos from MesosCon NA 2017 here: https://www.flickr.com/photos/pleia2/albums/72157689480391705

Chester Beatty Library and some whiskey

As I mentioned in my last post, at the end of August I traveled to Ireland with MJ for the second time. My first visit there in 2010 was pure tourism, and I hit all the major tourist attractions and historical sites and was able to visit with a local friend to learn about the Vikings, go to the Dublin zoo, see some Leprechauns, tour the Wicklow Mountains, take part in Guinness and Jameson tours, check out the Book of Kells, Dublin Castle and a couple cathedrals and visit Malahide Castle after feeding some seals in Howth. I even went to an Ubuntu release party. It was a whirlwind week, but I had a wonderful time and fell in love with Ireland on that trip.

It took me seven years to return and one of the first things I noticed was how much construction was going on this time. When I was there in 2010 the area was immersed in an economic downturn, but things had picked up considerably since then. The basic historic points of the city remained untouched, but between them were cranes and building of all kinds, particularly out where my husband’s office is down by Dublin Bay. Still, it was the Dublin I knew and loved, and my first order of business upon arriving on Tuesday morning off of an overnight flight was to visit some sights I hadn’t seen before. My jet lagged day began by making my way to the Chester Beatty Library, recommended to me by my friend Walt.

The Chester Beatty Library is now one of my favorite places in Dublin. As explained on their website:

Manuscripts, miniature paintings, prints, drawings, rare books and decorative arts complete this amazing collection – all the result of the collecting activities of one man – Sir Alfred Chester Beatty (1875-1968). Egyptian papyrus texts, beautifully illuminated copies of the Qur’an, the Bible, European medieval and renaissance manuscripts are among the highlights on display. In its diversity, the collection captures much of the richness of human creative expression from about 2700 BC to the present day.

Admission is free, photos are not allowed inside (though their online image gallery is nice), and it’s not a large museum, but it’s meticulously laid out to take you through a tour of what is largely a collection of religious manuscripts of various faiths. The illuminated manuscripts across various faiths were breathtaking, their Witness to the Word: Chester Beatty Library New Testament Papyri exhibit was fascinating, and I enjoyed seeing the mid-18th Century Astrolabe. However, the two things that stood out for me were the Qibla compass and the Mughal-era Indian collection of composite images. Walt told me about the Qibla compass so I made sure I specifically sought it out, I’m glad I did, I would have missed it if I hadn’t been on the lookout for it. As wikipedia explains, “a Qibla compass is a modified compass used by Muslims to indicate the direction to face to perform ritual prayers.” It’s a simple, yet clever device, which works, again consulting wikipedia, as follows:

To determine the proper direction, one has to know with some precision both the longitude and latitude of one’s own location and those of Mecca, the city toward which one must face. Once that is determined, the values are applied to a spherical triangle, and the angle from the local meridian to the required direction of Mecca can be determined.

I wish I had jotted down some notes about it while I was there, but just-off-a-plane brain wasn’t interested in taking notes. I have sent a quick note to the library through their contact form to learn more, fingers crossed that someone will get back to me with even the most basic details that accompanied the display.

Qibla compass at the Chester Beatty Libraryimage source (I wish I could be more specific as to source)

EDIT: They got back to me! Details:

Universal Qibla Finder
Made by Barun al-Mukhtari
Turkish text
AD 1738 (dated AH 1151)
Istanbul, Turkey
CBL T 443

Depicted on one half of this device is the Ka‘ba in Mecca and various near-by sites associated with the pilgrimage. On the other half is a map of Europe, Asia and Africa (regions north of the equator only), with numbers indicating the location of almost 400 cities, the names of which are provided in a colour-coded, numbered list beneath the map. A long pointer is attached at one end to the city of Mecca and at the top of the map is a small compass.

To determine the direction of Mecca and hence the direction of prayer from one’s current location, find the city in the list beneath the map, rotate the pointer until it points to the number (and colour) of that city on the map, and move the device until the compass needle points due north-south. When correctly aligned, the pointer will indicate the qibla, the direction of prayer.

As mentioned, the composite animal images (creatures whose bodies consist in whole or in part of human and animal figures) were the other thing that grabbed my attention. Specifically there was one of “A Peri Riding a Composite Lion” that captured the imagination. I knew nothing about the Indian tradition of composite animal art before visiting this museum, but seeing a collection together was really something. I stopped at the gift shop on my way out specifically to see if I could get a post card or a book with the composite lion image, and they didn’t disappoint, €7.95 later I came home with a coloring and postcard book of the images in their collection, including the lion!

“A Peri Riding a Composite Lion” circa 1700, Kashmir

I now consider the whole museum a must see when visiting Dublin.

Cultural enrichment behind me for the day, my next stop was a new whiskey distillery that I heard about, Teeling Whiskey. I learned there that the current iteration of the business launched in the 2015. However, the background of the brand, discussed on the wikipedia page and recounted during the tour, is interesting. In short, the Teeling family founded a whiskey distillery in 1782, but it sold and eventually shut down as a series of unfortunate events caused the entire Irish whiskey market to crash in the early 20th century and whiskey production move out of Dublin. After our history lesson, we were able to get up close to their on site distilling setup, including their big pot stills that the whiskey goes through for the traditional three-phase distilling process for Irish whiskey.

As we were tasting some of “their” three+ year old whiskey at the conclusion of the tour I joked about being pretty good at math when I asked “how can I be drinking a six year old whiskey when you were founded less than three years ago?” The history of their family explains, the current batches being sold under the Teeling brand actually came from the father of the founders, who founded the Cooley Distillery north of Dublin in 1987 and gave his sons a large collection of minimally aged whiskey. The results of these older barrels under the Teeling branding have already made their way stateside, but I’ll be keeping an eye out over these next few years for the ones with vintages older than 2015.

Teeling tasting

Teeling I did on my own, but a few days later was the final full day MJ and I spent in the Dublin area itself found ourselves concluding the evening with a stop at the Bow St location of Jameson. They have long done their distilling elsewhere, even in 2010 when I did the tour for the first time. However, this location is a nice tourist attraction that has a large bar and a tour. During my last trip it was more tourist-y and a bit kitschy, but I actually enjoyed it. We learned that week that they had recently redone their whole building and the tour, a perfect excuse to go back!

Fancy redone bar at Jameson

The tour is simplified now, much more refined with a greater focus on history and how the whiskey is made. They’ve also added several other tours for people who are more into Irish whiskey. I admit missing the old style tour a bit, but I’m a sucker for tourist stuff. The concluding tasting was enjoyable though, everyone gets to sample an American whiskey, Scotch, and some Jameson, to compare the styles and become our own experts at telling the difference. It was fun, and afterwards we all had a drink ticket for one final drink on the way out. Contrary to my typical “on the rocks” preference, my day called for a nice bit of Jameson, neat.

Our Ireland trip wasn’t all museums and whiskey though! Our trip to a transit museum and the long journey to the Cliffs of Moher are due in my next posts.