Your heart, your calories, your sleep, your data: How to extract your Fitbit data and make graphs using R

I opted for a Fitbit because they say “your data belongs to you”. But as it turns out, it is no easy task getting access to my own data from their main website where users have their personal accounts. Their data export feature is extremely limited and no way to download raw data. (By “raw data” I mean the numbers that I can use to do my own analysis and graphing.) Fitbit only provides data summaries and basic visualization. And currently, I don’t even see a way to extract my heart rate data, which really shouldn’t be that difficult for them to provide access.

I ended up spending a considerable amount of time messing around in the Fitbit API website. So I’ve decided to put together a step-by-step tutorial on how to get your data. And as a bonus, I’ll provide codes to generate heart rate time-course graph (like the ones in the last couple of posts). This requires no prior programming or scripting skills, but this assumes you’re a Fitbit user. And you will also have to be a bit adventurous and get comfortable using scripts to interact with your computer instead of the usual point-and-click with a mouse. This may be novel to some, but worry not. It’s easy!

(Note: Fitbit seems to keep changing their access protocol. This is the most up to date as of October 2017)

Setting up authorization in the Fitbit API

To get started, you’ll have to go to the Fitbit API, create an account, and pretend that you’re developing an application.

1. In main page go to Tools > “Register web app” and create an account
2. In the web registration step (Figure 1), enter the following information:
   • Application name and description (whatever comes to mind)
   • In application website and organization website, I entered https://github.com/ (some enter google.com)
   • For “OAuth 2.0 Application Type” make sure to select personal and make the callback URL “https://www.google.com/”
   • Select read&write and then click Register

     

     Figure 1

3. The above steps will create an application page with Client ID and Client Secret (Figure 2). Then click on the “OAuth 2.0 tutorial page”

   

   Figure 2

4. The first thing you need to do in the authorization page is to set up the type of authorization
   • Choose “Implicit grant flow” and choose the data you want access (I select all) and then click on the URL at the bottom.
   • This will open a new tab (Figure 3); select all and then click “Allow”, and this will take you to a Google page (i.e, your callback URL).

     

     Figure 3

   • Next will be “2: Parse response”. From the newly opened Google site, select the full web address and copy and paste it to the box under “Parse response”. This step will generate an authorization token for you (Figure 4).

     

     Figure 4

 

Getting your data!

Ok, so here is where I deviate a bit from the standard way provided by Fitbit. Fitbit uses a cURL command, which is basically a command line tool to download stuff from websites. A user may simply click on “send to hurlit” (Figure 4) and the website will run that command on your behalf. Then you’ll have to convince them that you’re not a robot, then they’ll give you the data via the web browser, then you’ll have to tediously copy and paste. Not efficient and not the best way to go about, in my opinion. But if you’d rather point-and-click, then sure. But I prefer using the command line.

[If you’re a Mac user, the command line is in Go > Applications > Utililies > Terminal and this will open the inbuilt Unix terminal. For a Windows user… er… ask someone. 🙂

In the terminal, enter the following code below. Let me break down the specifics of this. The “curl -i -H” is the function call to download data. Within quotes you enter your token that was parsed for you in step 4 above (like so… “Authorization: Bearer yedieyefi007yeyeverylonggibberish”). The code below is asking for heart data, from date 2017-10-07, at 1 minute resolution (which effectively is heart beats per minute) and from 8:45 AM to 10:15 AM (%H:%M format). Obviously you’ll need to set your own date and time according to your preference. [this code is longish and you can place your mouse on the code window and scroll right]

curl -i -H "Authorization: Bearer enter-your-token-here" https://api.fitbit.com/1/user/-/activities/heart/date/2017-10-07/1d/1min/time/08:45/10:15.json > mydata.json

 

This command is specifically to get heart rate. If you want other types of data, like your sleep, the parameters for this request can be found here.

This then saves the data as a simple text file named “mydata.json” to your computer. The data inside is in a format called json. You can open this file with any text editor to see what’s inside. Yay! But yes, Fitbit gives you the data as a json format instead of simple rows and columns that you can easily copy and paste to an excel file.

Saving your data to excel and plotting on R

Brief R tutorial.

What it R? R is a computer language and also an interface (sort of like your Unix or DOS terminal) where you can do all kinds of stuff! Go to https://www.r-project.org and install the latest version of R. Then open the R window and install the following packages: (1) jsonlite, (2) ggplot2. Installing a package is very easy. Type in: install.packages(“package name”) (Figure 5).

Figure 5

I will assume that the “mydata.json” file is in your home directory. You can move it to a different folder if you like, and when you open your R session, navigate to that folder. For example, if you moved mydata.json to a folder on your desktop named “MyFitbit”, in R simple type the “set working director” code:

setwd("~/Desktop/MyFitBit")

Then copy and paste the code below [this code is also long so again, place your mouse on the code window and scroll]:

library(jsonlite) ##loads jsonlite to your current session##
library(ggplot2) ##loads ggplot2 to your current session##
d <− file("mydata.json", "r") ###connecting to file; "r" mode is open to read in text mode##
d2 <- readLines(d)
d3 <- fromJSON(d2[16]) ####relevant line is #16, read that in ##
hr <- d3$"activities-heart-intraday"$dataset ##and convert to dataframe##
dim(hr)
write.csv(hr, "mydata.csv")###optional, if you want to save the data so you can open in excel##
hr$Min <- seq.int(nrow(hr)) #adds a column with minutes of workout##
head(hr) ##take a peek at data##
uplimmin <- max(hr$Min)
tiff("MydataGraph.tiff", width=10, height=8, units="in", res=600, pointsize=16)
ggplot(hr, aes(x=Min, y=value, color=value)) + geom_line() + geom_point(size=0.8) + scale_y_continuous(name="Heart rate (bpm)", limits=c(40, 200), breaks=seq(40, 200, by=10)) + scale_x_continuous(name="Time (minutes)", limits=c(min(hr$Min),max(hr$Min)), breaks=seq(0, uplimmin, by=10)) + scale_color_gradient2(name="Heart rate (bpm)", low= "#DBB165", mid="#2E604A", high="#D1362F", midpoint=120) + ggtitle("My heart rate (customize title)") + theme(axis.text=element_text(size=14), axis.title=element_text(size=16,face="bold"), plot.title=element_text(size = 16, face = "bold"))
dev.off()

The ggplot code looks complicated (on line #13). Let me break it down so you can customize if you want:

• scale_y_continuous(name=”Heart rate (bpm)”, limits=c(40, 200), breaks=seq(40, 200, by=10)): Here I’ve set the scale from 40 bpm to 200, you can changes those numbers
• scale_color_gradient2(name=”Heart rate (bpm)”, low= “#DBB165″, mid=”#2E604A”, high=”#D1362F”, midpoint=120): I used those hex color codes; you can change them, or even simply writing “red”, “blue”, “black” will work
• ggtitle(“My heart rate (customize title)”): this sets the title for the graph. Name it anything you want.

[Update on 19 Oct 2017: quick note of code line #5, the “d3 <- fromJSON(d2[16]”). The [16] is because my json data has 14 lines of header information followed by one empty line and the actual data starts only at line 16. Someone alerted me that their data started on a different line. So yours may be different. The best thing to do is to open the file in a text editor (e.g., Mac’s TextEdit or Textwrangler) and see which line, as in which row, the data starts in your file and change the number in fromJSON(d2[16]. The line that data starts will look something like this {“activities-heart”:…. etc.]

This will create a tiff file with your graph and the file name will be “MydataGraph.tiff“, but you can name it what you want in the code above (in tiff(“gobbledygook.tiff”…)).

There we go.

[please excuse typos and errors; not proofread and author makes lots of typos]