I love this! I'm a T1D as well, and really appreciate N=1 work understanding our own bodies with this (and other) conditions.
Do you mind if I share with an open healthware group that I'm a part of?(https://healthware.oshwa.org/) This is right up their alley
Feel free to share! I am currently testing Mistral's newly released TSFMs (Time-Series Foundation Models) on my blood sugar the results of which will be in the repo in the next couple of days.
89% TIR, well done! Cool to see the research, I’m surprised there aren’t more in-depth n=1 studies out there from statistics/data-savvy diabetics. I built a small dashboard exactly collating carb, insulin, sleep, and movement data and trying to coax out useful correlations to help manage. Some interesting insights, but clearly I should eat less and move more. Keep up the good work, and thanks for sharing!
Ironically, did the same thing and ended up with the same answer.
From wearing a CGM, whoop, using an impedance analyzing scale (I know, I know), blood tests - all the data shows pretty much the same correlation. I mostly did it for fun, but also because I wanted to "figure out how different things impact my body".
While yes, there were some interesting discoveries like:
- Some things raise my blood glucose more than I'd expect, pasta for example - those were the things I usually "knew" and foods I'd avoid eating.
- Sleep quality only had correlation with amount of activity/time I went to sleep, so my food and caffeine choice didn't impact it much.
The results were pretty much:
Eating bad and moving less is bad for you. Eating good and moving more is good. What your intuition tells you is healthy and good for you, usually is. What it says isn't, usually isn't.
Could have saved me some time and money, but at least I had fun.
Yeah I'm thinking of adding more (especially recent) models and compare their published metrics with those of mine to see how they hold up. Then I'll likely post about it here again. See you then!
Incredible work! As a T1D I've been gathering as much data as possible (blood glucose, insulin-on-board, food consumed, heart rate, skin temperature, etc.) for exactly this purpose. Can't wait to dive in!
BG forecasting and insulin response has always seemed to me to be such a personal thing (everyone has such a different response based on the various factors, and there are _so_ many factors) that the one-size-fits-all models in CGMs seemed the lowest common denominator. We live in an age where personal data collection is easier than ever (with wearable BLE hardware) and timeseries forecasting models are better than ever, making this research even more valuable.
Thanks, N=1 training is really the way to go. In my case training on large N >> 1 (i.e. metabonet dataset) actually hurt the performance of the models predicting my own blood sugar.
You don't even need a datacenter GPU. I trained all of my models on a laptop RTX 4070.
Exercise can have a substantial effect on glucose. Have you considered including physical activity as an input feature, e.g. from a smartwatch (heart rate, steps, activity intensity)? Or is that data simply not available?
I absolutely hate exercise. I mean as a machine-learning guy it is the hardest quantifiable feature and yet it causes large swings in blood sugar. I haven't added exercise as additional input yet because I'm still searching for a better way to quantify it.
Is heartrate not a decent first approximation? I wouldn't have guessed it would be the best way, but I would have guessed it would be a decent first attempt and would increase model performance.
Unfortunately no, well at least not in my case. My heartrate gets higher before I wake up, and incidentally this is also the time when there is a higher likelihood of getting high blood sugar.
Regarding exercise, my blood sugar gets higher during anaerobic exercises but lower when doing aerobics.
At the moment I consider heartrate a confounding variable rather than a good predictor of BG. I think it could work for sedentary populations for whom higher heartrate correlates well with some kind of aerobic activity.
It's so cool. I appreciate the detailed architecture doc. I wonder why the time of day is getting predicted rather than provided in data. Did that perform worse? Or is it to explicitly account for people with uncommon schedules?
When I began this project my model was indeed being provided time-of-day features (they were sinusoidal like in the published literature). Then one day I decided to remove it to see what happens. The accuracy didn't degrade so I thought hmm maybe it's because the model can *infer* time-of-day from past blood sugar alone? So I attached a time-of-day head to test it and it worked!
Interestingly I have noticed that I get sleepy when the model's predicted clock gets close to midnight rather than the actual clock when I travel between time zones.
Extremely interesting, thanks for sharing!
Have you looked at Timestral at all by any chance?[0]
I'm only dipping my toes into time series models, for somewhat similar reasons.
TSFMs are really really big. My motivation for training smaller models is because I want to run them locally on my smartphone. But yes, I plan to add TSFMs to the github repo soon.
Is there anyway way to use xdrip or nightscout or similar to move the data into your (android) app?
I already use the official Freestyle Librelink app plus jugglucoNG (which btw is amazing if you don't use, has a lot of nice stats), and the latter can already export its data streams.
Side note, I'm very happy to see more (diy) work in this field (ever since AAPS I think). #wearenotwaiting !
(Hashtag context for non T1Ds: diy software, unsurprisingly, has been leagues ahead in capabilities than "official" software.)
You can copy the data from the app to nightscout, but I haven't added the reverse direction. My app allows me to read from multiple CGMs simultaneously (I did this because different CGM models have their own algorithms and you don't want your machine learning model to be fitted to one particular algorithm).
I do a lot of reverse engineering to connect my app directly to the CGM sensors, bypassing official vendor solutions (they suck anyway).
cool to find this project. I'm implementing literally the same thing recently. I'm using exact same insulins. My setup is a bit different and a bit behind you in terms of model quality.
I run pretrained models on raspberry pi, so raspberry acts as server. I use dexcom, garmin mcp to gather all data. Android phone acts as dumb client. I have create a custom app to log carbs, insulin injections, predict insulin doses from a food photo.
There are also some nighty jobs to recalculate insulin sensitivity, retrain model, generate daily report.
I started logging carbs and insulin doses few weeks ago, so currently there is not enough personal data to train prediction model. I use oref0 algorithm for predictions currently. As a starting point i have created prediction model solely on glucose values. Training on other people data, did not helped me to make predictions accurate.
I also didn't have enough personal data to train on (transformers are very data-hungry), so I created a huge synthetic dataset to help me with that. I use that for pretraining and then plug in a LoRA that got trained using my personal data.
I spent a long time trying to build predictive models that use complete and accurate carbs/protein/fat macronutrient data as well as insulin delivery. I found that there were enough non-linearities and other unmodelable factors at work in the BG data that led to very poor predictive ability. Considering protein & fat had a nil-to-negative effect on the models.
It's possible that accounting for heart rate as well as carbs & insulin might have better results, with enough data.
Yeah I was using the macro-nutrient approach as well, but then switched to simply providing GI (glycemic index) alongside the carbs in grams and found out that there was practically no difference between the two.
During training I artificially wobble the carb amounts so they are more like guesstimates than true carbs. This also partially fixed the quantile bands (before that they were too narrow).
Helping hands are always needed. You can test the models on your own data locally and tell me how they work for you. I have 2 kinds of models in the repo: those trained on the metabonet dataset which consists mostly of AID, and mine which is MDI.
Have you ever thought about fixing the underlying issue instead? Maybe it makes this whole project obsolete. I know T1D can't be fixed, but using ketogenic diet should have much smaller peaks/lows & more stable blood glucose in all cases.
Removing carbs will remove most (not all) need for insulin.
I have done keto for 4+ years, and have done ~2 years epilepsy keto, where the measurement was to have blood ketones 2+ & glucose 4 (mmol/l) , 1 hour after wakeup.
Epilepsy-keto is in the same realm as this, but even harder, because you need high ketones & low glucose. You're literally eating 10-20g carbs a day. The maximum that I ever ate while maintaining epilepsy keto was 50g carbs/day, and it was only possible by doing heavy exercise (120 squats with your body weight in the bar).
I don't have experience of epilepsy so wouldn't assume about its challenges, but do have T1D. I've done some long periods of strict keto, and it does have some big positives, but I don't find it a great long term solution.
From wearing a CGM, whoop, using an impedance analyzing scale (I know, I know), blood tests - all the data shows pretty much the same correlation. I mostly did it for fun, but also because I wanted to "figure out how different things impact my body".
While yes, there were some interesting discoveries like: - Some things raise my blood glucose more than I'd expect, pasta for example - those were the things I usually "knew" and foods I'd avoid eating.
- Sleep quality only had correlation with amount of activity/time I went to sleep, so my food and caffeine choice didn't impact it much.
The results were pretty much: Eating bad and moving less is bad for you. Eating good and moving more is good. What your intuition tells you is healthy and good for you, usually is. What it says isn't, usually isn't.
Could have saved me some time and money, but at least I had fun.
BG forecasting and insulin response has always seemed to me to be such a personal thing (everyone has such a different response based on the various factors, and there are _so_ many factors) that the one-size-fits-all models in CGMs seemed the lowest common denominator. We live in an age where personal data collection is easier than ever (with wearable BLE hardware) and timeseries forecasting models are better than ever, making this research even more valuable.
Thanks so much for publishing your work!
You don't even need a datacenter GPU. I trained all of my models on a laptop RTX 4070.
Regarding exercise, my blood sugar gets higher during anaerobic exercises but lower when doing aerobics.
At the moment I consider heartrate a confounding variable rather than a good predictor of BG. I think it could work for sedentary populations for whom higher heartrate correlates well with some kind of aerobic activity.
Interestingly I have noticed that I get sleepy when the model's predicted clock gets close to midnight rather than the actual clock when I travel between time zones.
[0]: https://mistral.ai/news/timestral/
Is there anyway way to use xdrip or nightscout or similar to move the data into your (android) app?
I already use the official Freestyle Librelink app plus jugglucoNG (which btw is amazing if you don't use, has a lot of nice stats), and the latter can already export its data streams.
Side note, I'm very happy to see more (diy) work in this field (ever since AAPS I think). #wearenotwaiting !
(Hashtag context for non T1Ds: diy software, unsurprisingly, has been leagues ahead in capabilities than "official" software.)
You can copy the data from the app to nightscout, but I haven't added the reverse direction. My app allows me to read from multiple CGMs simultaneously (I did this because different CGM models have their own algorithms and you don't want your machine learning model to be fitted to one particular algorithm).
I do a lot of reverse engineering to connect my app directly to the CGM sensors, bypassing official vendor solutions (they suck anyway).
Do you have a GitHub repo or something btw?
It's possible that accounting for heart rate as well as carbs & insulin might have better results, with enough data.
0xdeadf1sh: good luck with your quest!
During training I artificially wobble the carb amounts so they are more like guesstimates than true carbs. This also partially fixed the quantile bands (before that they were too narrow).
JK, as the spouse of someone with T1D I really appreciate seeing this!
Removing carbs will remove most (not all) need for insulin.
Epilepsy-keto is in the same realm as this, but even harder, because you need high ketones & low glucose. You're literally eating 10-20g carbs a day. The maximum that I ever ate while maintaining epilepsy keto was 50g carbs/day, and it was only possible by doing heavy exercise (120 squats with your body weight in the bar).
What are the issues for long term?