A side project of mine that I’m working on at the moment is StackCompare, an app for StackExchange users to compare their reputation and badges to that of their friends. One feature I wanted to add was a graph of reputation over time.
Step One - Data Aggregation
First things first, get the data into some sort of database. The data is a time series (a set of tuples of the form (timestamp,data)), so my thoughts immediately went to setting up my own OpenTSDB instance. However, where possible I’d rather use a hosted solution at this stage of development, and some googling led me to TempoDB, a hosted time-series database service. Currently Tempo seems quite early-stage (a warning sign to me is lack of any mention of pricing…) but works quite nicely, with decent documentation and a Python client.
classCommand(BaseCommand):help='Grab reputation for all users and their friends, and store in TempoDB'defhandle(self,*args,**options):s=stackpy.Stackpy(settings.STACKEXCHANGE_CLIENT_KEY)user_ids=_get_all_user_ids()users=s.users(user_ids).itemstempo.write_reputation('stackoverflow',users)
As StackCompare is currently a Heroku app, it was trivial to hook up the Heroku Scheduler to run this every 10 minutes.
Step Two - Data Extraction
Getting things out of TempoDB is equally straightforward:
All that was needed to finish it off was some short Python code to massage the data from TempoDB into the right format for Flot (slightly paraphrased):