Stay up-to-date with your favorite Github repositories with Porter.io

A problem and a solution

As developers, we use quite a lot open source code in our projects and often have to absorb massive amount of information everyday to keep updated with the ever changing open source world. Some of us even have HackerNews obsession and feel irresistible to refresh its front page every few minutes in order to be among the first ones to know the next cool thing.

We used to rely on RSS but it faded away from mainstream. There's always something new popping up everyday and RSS doesn't help much with a static reading list. The major technology sites and forums are flooded with all kinds of information. It’s a pain to filter them with our own eyes. In the financial world there’s the Bloomberg which collects, filters and analyzes information for the clients based on their interests. Why can’t there be one for the open source world? In an open source way?

We started by integrating Github API and collecting as much historical data as we can. As Github is already one of the de facto center of the open source community and more importantly that we are all hardcore Github users, we feel it’s natural to embrace its workflow and “take advantage” of its available data. That’s why our service is project oriented, or to be more specific, Repository oriented. We believe that nothing represents one’s technical interests better than his repos, no matter starred/watched/forked or his own. With that information in hands, we are able to model a personal filter for every Github user with ease. And, it simplifies the user experience a lot too. No more endless selecting and deselecting to build a list of interests. Just sync! Thanks to Github.

Once the subscription list is ready, we will generate daily digest emails based on the repos of interest. Porter.io has a smart algorithm to automatically collect the news or discussion about your repos on the internet. Currently we are monitoring news submitted to HackerNews and Github repositories. More news source and repository source will be added in the future.

Who is Porter.io for

Open source user

We’re releasing Porter.io as a free service for everyone so all developers are welcome to use our service. But as you star more repos at Github, you will receive digest email more frequently and find our service more useful since a short subscription list will often result in only a few or no news discovered. Of course you could always sign up with us first and grow your subscription list afterwards.

We are also planning on releasing a recommendation feature, which would help you discover more useful repos based on your interests. At that time, even someone just entered the open source world will benefit from our service.

Open source author

By analysing the growth graph we provide, open source authors could get better understanding about how your project grows and what brings users to your project. By subscribe to your own projects, you’ll never miss an update when someone is writing about your project.

Discover, analyze and subscribe

Porter.io is very easy to use. By adding ‘http://porter.io/’ before the url of the Github repo you want to know about, you’ll instantly access it’s growth graph and related news list. It’s very useful when you found a new project and wanted to learn more about it.

Subscription can be setup by simply login with your Github account and you’ll begin to receive email digest when there’s any news page related to your subscribed repos discovered.

More technical details

The crawler

We built two Python based crawlers with Gevent which monitor every change of HackerNews and Github. The HackerNews crawler will analyse every page submitted and associate it with any Github repository found in that page. The Github crawler will import all the star and fork events which is the source of the growth graph.

The website

The website is built with Django and Angular.js. Two favorable projects shared by a lot of developers in backend and frontend development.

The platform

We host Porter.io on Google Appengine and Google Compute Engine with Docker. We’re planning switch to Google Container Engine when Kubernetes become stable.

Suggestion and Feedback

We value any feedback from anyone. If you have some ideas about Porter.io, please reach us by twitter or email. With the help from our users, we could build a better product for everyone.

Interested? Login with Github to setup your subscription list.