What’s brewing and cooking in your open source community? (Part 2) — Shilla Saebi Comcast

What’s brewing and cooking in your open source community? (Part 2) — Shilla Saebi Comcast


♪ [music] ♪ Okay. So, this is part two of the talks that you
just heard from Alison. She talked a lot about executive buy-in,
especially leadership buy-in, as well as branding,
and dove into internal metrics. I’m going to go ahead and dive a little
bit deeper into the metrics that we put out, internal ones for our
communities and external. A little bit about me before I begin,
my name is Shilla Saebi, and I’ve been with Comcast for a little
bit over nine years. I am an open source program manager. Prior to this role, I was on the cloud
team, and I was an operations engineer. You might be wondering why Comcast has an
Open Source Program Office or what they’re doing with open source. So, I will give you all just a little
recap of the journey that we’ve been through. I worked with a number of folks that
have been at Comcast longer than I have. They’ve been there for much, much longer,
and they’re open source contributors. And I reached out to a bunch of folks from
different orgs as well, to find out, when did open source really
begin at Comcast? 2006 was the date that was consistent. We know that in 2006,
we were large consumers of open source. In 2012, we launched our first two
open source projects. And in 2013, the Open Source Advisory
Council was formed. The Open Source Advisory Council consists
of folks from our legal team as well as security and engineering teams,
well-respected engineers across the company. They are a group who decide whether
open source contributions should go through or should not go through. And so, 2013 is when that was launched. In 2015, we were now large
contributors to open source. We won the OpenStack Superuser Award. In 2017, the Open Source
Program Office was created. And that’s where you’ll see we joined a
bunch of different foundations, and did sponsorships like the Linux Foundation,
CNCF, Apache Software Foundation, etc. We also launched comcast.github.io,
which is a showcase of our open source projects for external community folks. Fast forward to 2018, last year,
we had our first open source conference that was available for,
not only Comcast employees but to the community. We had about 70% external attendees and
30% internal to Comcast. And we also launched the Open Source
Fellowship pilot. This is a pilot program that was launched,
specifically for our engineers to work on open source contributions. We have seven open source projects that
we’ve selected, that we know that we’re large consumers of, and our engineers get
to work 20% of their time, originally, it was open source Fridays,
but it’s 20% of your time, whenever that may be,
to work on open source, and that is 100% supported
by our leadership. So, we’re really proud of that. And then in 2019, which is today,
we have several hundred open source projects. Our team has grown tremendously. And we’re looking at putting more strategy
around open source. We also have a partnership with our talent
acquisition team, so that when we go to conferences, we’re showing up with a
strategy and organized. So, the team that I work on,
I work in the Open Source Program Office. Like I mentioned, we opened
our doors in 2017. We support over 7,000 developers. We have 224, maybe more as of today,
repos on GitHub. And we are a seven-person team. Just in the last month,
we’ve hired two folks. We have two software
engineers on the team. We have two program managers. We also have our leader
as well as a summer intern that’s come on board to help us with metrics. So, what’s brewing in our
open source communities? So, a little bit about how we kind of got
into metrics, why they’re even important. So, originally, once my colleague and I
started on the team, we knew that open source was happening,
we knew that people were contributing to open source, but we didn’t really know
who those teams were or where it was coming from, and there was no
organization around it. So, what we started doing was,
we started meeting with different engineering teams each week. Some weeks, we were meeting with two or
three teams, to the point where we were exhausted from meeting with so many
engineering teams. You know, I don’t know exactly how many,
but I want to say maybe 30 or 40 engineering teams, and we have so many
more that we haven’t even touched. But once we started meeting with these
different teams, we started getting a lot of open source contribution requests. We’re telling people, “Hey,
we love what you’re doing with open source, please just come to us,
and talk to us about it. We want to partner with you. We don’t want to necessarily
police you or anything. We want to make sure that you’re using the
right licenses and that we’re also showing gratitude when you are doing open source
contributions, and that we know what’s going on.” So, the communities that we report out on. We report out on our internal community,
achievements, and progress that we’ve made, and also on our external
communities. Our internal communities are the 7,000
developers that I mentioned, the thousands of developers, plus,
we also have other program managers, project managers. It’s not just necessarily software
development engineers. We have a ton of people in the
open source community. Also, the open source projects that we put
out, we want to make sure that we’re watching what’s happening
with those communities. VinylDNS is a one-stop shop portal for
DNS configuration management. Trickster is an open source dashboard
accelerator that plugs into Grafana or Prometheus. Kuberhealthy is easy synthetic
testing for Kubernetes. So, these are some examples of some of the
projects that we’ve put out. So, we want to look at
what’s happening there. So, the internal community
metrics that we track. We look at who our top open source
contributors are in the last year. We want to reward them. We also showcase them on our blog. We also showcase them on this
exact report that goes out. Our community growth is another internal
metric that we look at. The place that the entire internal
open source community mainly hangs out is in Slack. And so, we want to make sure that, that
community is growing. We’re making people aware that there is a
place to congregate, and hang out, and ask questions, and chat about
open source. Also, our internal presentations. We also do a bunch of internal
presentations across the company. We look for opportunities to speak with
those engineering teams that we’ve met, because now we’ve built this partnership
with them, we work with them. And if we know that there’s an event
happening, we’ll ask them, “Hey, do you all have, like, 20 minutes
in your schedule? We’d love to talk about the
open source program.” Incentives and gratitude that is given to
the OSPO from our developers, we definitely collect and share our kudos
with the team. We started doing that a few months back
and actually reported out on it. And it was really nice to see all of the
thank yous that we’re getting from engineers for simplifying the whole
open source process. Prior to our team joining,
the engineers had to kind of talk with legal, and go through why something
should go open source. And that wasn’t really the best
experience for them. So, it’s really nice to see that they’re
thankful for the work that our team is doing. And we do the same thing for them. So, anytime any open source contributors
make a contribution, we make sure to thank them. We have a number of
different ways to do that. We have portals. We do handwritten notes. We do cards, but we track. And then the internal events that we host. We do open source days twice per year. Last year, we went to Seattle and Denver. And the reason why we have to travel for
them is because they asked us to come. So, they asked us to come and do a mini
open source day. And so, we came because Philadelphia is
where our headquarters is. And we have a very large auditorium there
where we do a lot of conferences. But the other remote offices asked for us
to come out, and we did. This year, we just did one in Sunnyvale. And then we have another one
coming up in Philadelphia. And then the number of open source
contribution requests that come in. So, we look at those and that number,
we’re watching the growth. And it’s really good to watch those
because that’s when we can actually justify that we need more
people on our team. And then the type of open source
contribution requests that are coming in. Are they bug fixes? Are they full-blown projects? Are they documentation? We look at that. And then here are some examples of the
community metrics that we track. I’m just going to show you all exactly the
slide deck that I pretty much put out twice per year. And I won’t include every single slide,
because in the interest of time, but I’ll include a few so that
you can get an idea. So, the Slack community growth, here, you’ll see that our community,
we have about 900 people in our Slack channel today. I’ve encouraged people on our team to just
put it in their email signature. I have it in mine, I just say, “Hey,
join our Slack channel if you have questions.” Whenever people DM me,
I’ll reach back out to them, and I’ll tell them, “Hey,
we do have this Slack channel where you’re more than welcome to ask that question.” And so, we’re continuing to grow that
community, and it’s something that we’re watching. Internal Presentations. These are the ones that I mentioned
that we do. We have a showcase series. We do an open source educational series. I did a recent webinar on innersource. We had a very good turnout. Innersource is a hot topic. So, back by popular demand,
I did another presentation on innersource at a internal event that we did. So, we want to make sure that we’re
continuing to raise awareness about Open Source Program Office, what’s happening. And so, these are the
internal presentations. And then the events that we help organize
internal and external, shameless plug, June 20th in Denver, we have an open source conference that’s
open to the public, and it’s free, so you’re more than welcome
to come if you’re around. But that’s a great example of an event
that we’re helping organize. I’m also the meetup organizer for the
Prometheus group in Philadelphia. And also, we do a number of other events,
like I had mentioned, the internal ones. The number of open source contribution
requests that are filed. Here, you can see that there’s a
tremendous growth since 2017, since we opened our doors. And we’re projecting to go
even higher this year. And this is exactly what we used to show
that we need more help on our team. So, we now, just a month ago,
brought another open source program manager to help us with these contribution
requests, because we only had one person working on these. So, the external community
metrics that we track. Pull requests to the open source projects
that we maintain. This is super important to us. And we’re not micromanaging or policing
anyone, but we want to make sure that the open source projects that we’re
maintaining, if there’s a pull request that comes in, it’s not
sitting there for a year. And also, we help coach our engineers on,
you know, how to respond to things in the open source community. We do a lot of blogging. We let folks know. Sometimes people, it’s their first time
working in open source, and so they don’t know. You know, there could be a pull request
that sits there for six months, and that other person on the other end has
no idea what’s going on. So, we want to make sure that, that
kind of thing doesn’t happen. Also, outside contributors to the
open source projects we maintain. It’s really important to have diverse
committers, diverse contributors from different companies,
different areas, etc. And they’re not just Comcast-heavy. So, that’s something we definitely look
at, especially for the projects that we think are going to be really
useful to the community. Also, most starred, most forked,
and the newest open source projects we maintain. We understand that this doesn’t
necessarily measure the popularity of a certain project. But there are some communities that
require a certain amount of stars in order to be incubated into, for example,
a foundation or something. And so, it’s something that we look at,
and we use the GitHub API to do that. And that information’s actually on
comcast.github.io as well. It’s in the statistics at the
bottom of the page. Blogging. We have an engineering blog that is mainly
funneled through our corporate communications team. We work closely with them
to get stuff out there. We don’t blog all the time,
but we blog when we have huge announcements, like, let’s say,
“Trickster is now open source.” We’ll talk about something like that. If we have a whirlwind tour of number of
open source conferences, and we have a bunch of updates,
we’ll write a blog post about that. Recently, we’ve been doing blog posts on
opensource.com. They’re super helpful. Great, great team to work with. We were able to push out a couple
of posts through them. So, we’re also watching to make sure that
we’re continuing to put content out. And then also outside press pickups. I have an alert setup in Google. Anytime Comcast, Open Source,
anybody blogs, or writes anything, a news press, whatever,
I get a notification. So, that’s something we look at. We love to see positive press come out. And we think that giving back to the
open source community, and sharing our use cases, and putting projects out is
definitely helping us and giving us a lot of positive, positive feedback
in the open source community. Sponsorships with open source foundations,
that’s another thing that we look at. We want to make sure that we’re on top of
what’s happening, where we’re actually sponsoring,
where the money’s going, and then where it should go next. So, we talk to our engineers. We have a survey that we put out,
and we say, “Hey, here are the open source foundations and projects
that we’re sponsoring. Is there something we’re missing? Is there a community that you work with or
you’re a huge consumer of that we should be looking at?” And so, that’s really helpful. So, we track all that. Our external presentations, today counts
as an external presentation. Our team goes out to a number of different
places like FOSDEM, All Things Open, CHAOSCon, Open Source Summit, etc. We definitely track those. And Twitter engagements. We look at the impressions, the clicks to
see, especially if we have like a big event going on, and we want to make sure
that we’re projecting that message out to the community. We look at that kind of thing. And then lastly, traffic to
comcast.github.io, which is our public facing and open source website. So, a few examples of the
external community metrics that we track. Again, I won’t go through every single one
of those, but I’ll show you just a few examples of what we report out on. So, Trickster, that was the open source
project that we recently put out to the community,
and about 70% of the contributors are non-Comcast employees. Kuberhealthy, similar. This was actually the more recent
open source project that we published. About 50% of the contributors
are outside of Comcast. And then about 80% of the contributors for
the Apache Traffic Control project, which is now a top-level project,
are now outside employees. And I must say that once you donate your
project to a foundation, you do have an advantage of getting the
community from that foundation or from that project. So, that’s what it looks like for these
three. Published blog posts. We have eight so far. We have an editorial calendar. And so, we make sure to track it. And then OSPO public speaking engagements. It looks really low right now. And that’s because conference season
hasn’t hit us too hard yet. Around September through December
is when it will be crazy. And so, that number will likely spike up. And I’ll also include that we do put, and I don’t have it here,
but we also put out a highlight deck that lists out all of these things, so that
people can see. It’s not just, “Oh, they went to nine
public speaking engagements.” We actually list out what those speaking
engagements are. And then, like I mentioned,
the most starred, most forked, and our overall stats,
the total members of our overall stats look really low. We’re actually have hundreds of members in
our Comcast GitHub organization. However, by default, it’s set to private once you
get added to an organization, so we’re working on talking to folks and
see if they want to switch it to public. That’ll really help us with our metrics. And then this is the comcast.github.io
website traffic, and this is using Google Analytics. So, we’re really looking at the total
number of clicks, and then the difference between last year and this year,
and then figure out how we can grow that. So, as you can see, our community metrics
are a big mix of internal and external community. So, what’s next for us? Automation, automation, automation,
times a million. So, right now, what I am doing to get
these metrics are, I use Twitter, I use Jira. I go in, and I search out
through the filters. I also use GitHub API. Because the GitHub API has a time limit on
how much data you can get, we’re using another tool
called GitHub Archives. And that’s what we’re using to get the top
open source contributors from Comcast in the last one year. And also, that’s what we’re using to get a
number of other data points for the open source projects that we put out. Luckily, our summer intern is here to help
us with our metrics. We do have a dashboard, in-flight. And right now, it includes our top
contributors, as well as some community health metrics. But ideally, we’d love to plug the
Jira API, Twitter API, and get all that information into a one-stop
shop, and then have everybody just be able to access it at any time, transparent. We don’t want to have to put out some
report, you know, twice a year. We want anyone from our community to just
be able to click in and see what’s happening. And with that, thank you. ♪ [music] ♪

Leave a Reply

Your email address will not be published. Required fields are marked *