![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
I'd like to talk about a serious shortcoming of the medical system in treating a category of illnesses, and a simple, self-experimental approach that allows you to potentially treat these illnesses cheaply and easily.
----
Introduction:
"Diffuse Conditions" is a term I made up to describe health conditions that have numerous potential causes and numerous potential treatments. These conditions include insomnia, allergies, irritable bowel syndrome, depression, and anxiety.
Diffuse conditions are hard to treat via the medical system. Due to the economic, research, and procedural constraints of the medical system, patients are often steered toward a narrow set of potential treatments. Moreover, these treatments are generally expensive drugs or procedures rather than simple practical advice. In addition, because of the prescription system, patients are only able to try one potential treatment per doctor visit, and since diffuse conditions have many potential cures, that potentially means a lot of doctor visits to find one that works. In addition, the diffuse conditions are sometimes bad enough to affect a patient's quality of life, but not bad enough to merit "medical attention". Doctors are generally focused on fixing people when they are broken as opposed to tuning them for optimal function. Basically, all this means that someone with a diffuse condition is likely to have a lot of difficulty finding a solution, and will end up spending a lot of money (either directly or via the tragedy of the commons system known as health insurance) trying to find a solution. At this point, healthcare is already up to 17% of our GDP in America, so we need to figure out a cheaper approach. I believe medical experts can be useful in treating diffuse conditions, but they need to empower patients to engage in a highly iterative cure discovery process.
The solution, I believe, is to data-mine your own life. This is a five-step process.
1. Get quantitative tools for measuring your performance.
2. Try various interventions.
3. Track performance and interventions over time
4. Look for correlations to suggest ways you can improve your condition
5. Share your findings online so that others can see if your successful interventions work for them.
-----
I'm going to provide a personal example of this to show how it can be done:
Sleep videos:
I've always had trouble with insomnia. For a long time, I just ignored it as a fact of life. In my early twenties, I decided to finally do something about it and get a sleep study. I went to one of the best sleep clinics in the country (Stanford). They found some minor issues (sleep apnea so mild that it could not be designated "mild sleep apnea") but nothing medically actionable. Unfortunately, that gave me little to act on. A sleep study requires you to sleep in a hospital with an unbelievably large amount of equipment attached to your head. Sleep studies are so expensive that any given person is only likely to do one or two of them. So unfortunately a sleep study is not a good way of determining anything useful.
I got the idea to use a cheap near-infrared video camera (Sony sells analog "nightshot" video cameras that can see in the dark) to do my own sleep study at home by doing time-lapse recordings of myself sleeping. I ended up with fascinating videos like these, which show the range of strange things that happen during the third of our lives when we aren't conscious.
Here's what I learned and did about it:
- I move around a lot while I sleep. Limb movements were documented on the sleep study. They appeared to correlate either with my posture (specifically, lying on my back) or with dreaming. The limb movements appeared to be impacting my quality of sleep as they appeared to often line up with or precede awakenings as seen on the sleep study. By looking at several night, I was able to see that the limb movements most often appeared when I was dreaming, regardless of posture. Also, as any good scientist knows, it's good to have a control group for comparison. As a result, I taped myself sleeping next to three different people, and found that I moved around substantially more than any of them did.
- If I'm sharing a bed, it needs to be king-sized if I am to sleep well. The video shows many instances of sleep-disturbance ping-pong, in which one person shifting causes the other person to shift, and that's in a king bed. In a smaller bed, it's even worse.
- If I'm sharing a bed, everyone needs their own quilt. Quilt-sharing dramatically worsens sleep disturbance ping-pong.
- Morning light dramatically lowers sleep quality. The morning light stimulates production of hormones to help me wake up, but this is not useful if I'm going to bed many hours after sunset. As a result, I started using heavy curtains to help myself get a full night's sleep.
- I don't have Restless Leg Syndrome. Based on the movements, I thought I might have it, so I got a doctor to prescribe the two most commonly used RLS medications. Both of them made my sleep worse.
- Posture interventions didn't change my quality of sleep. It appears that I need to spend some time sleeping on my left side, my right side, and my back. If I choose one particular posture to fall asleep in, I will spend most of the tail end of that night in the other two postures. If I force myself into a particular posture (eg by wearing a shirt and stuffing a pillow into the back), I fight it intensely. I think this may have to do with circulation; no part of my body likes to be compressed for long periods of time. I dislike extended sitting when I'm awake.
- Anti-apnea technologies don't help reduce the movements. CPAP is loud, messy, and obnoxious, and actually made the movements worse.
----
Sleeping pills:
I also did an evaluation of just about every type of sleeping pill I could get my hands on, and rated how effective they were at getting me to sleep and how much of a "hangover" effect they have the next day. I evaluated them via self reports, and sometimes with the videos, for how much they affected my sleep quality. Your results may vary. Here's what I found:
- Most over-the-counter sleeping pills (Tylenol PM, Nytol, Sominex, Benadryl) are bad. They tend to be based on the antihistamine Diphenhydramine HCl. While they are great at getting me to sleep, they worsen apnea (probably by relaxing the throat muscles too much) and they seem to make me stupider the next day. I was excited when earlier this year, a study came out showing that my "it makes you stupider" observation isn't just anecdotal.
- Alcohol (even a single drink) is a bad sleeping pill.
- Valerian was okay (decent effectiveness but some hangover), as was Lunesta.
- Melatonin was best for everyday use, and it's available over the counter. Your body produces it naturally, and you're probably already deficient in it if you are around bright lights before you go to sleep. I found 1mg was sufficient to do the trick. At >= 3 mg, I started to have some hangover effect the next day and some habituation issues (making it harder to go to bed the next night without it). However, at 1mg, I had very little hangover. I now tend to take it if I haven't been able to fall asleep after 45min.
- Ambien is useful as a "nuclear option" when I'm taking a transatlantic flight or on a packed overnight bus. It has some hangover, but it's shockingly effective. It has the bonus effect of helping reset your circadian rhythm to a new time zone if you take it at the bedtime you desire for the new time zone. Ambien has been known to have some substantial side effects, like sleepwalking, in certain people. In addition, before putting you to sleep it provides you with a feeling not unlike drinking an entire six-pack of beer in one sitting. I wouldn't be surprised if it eventually ended up as an illegal drug.
----
Quantitative interventions
While the above research yielded a lot of gains, I was able to learn even more when I started using automatic sleep-tracking technologies like the Zeo. Zeo uses an EEG headband to monitor your brainwaves. It can determine if you're awake, dreaming, in light sleep, or in deep sleep, by looking at them. From there, it comes up with derivative measures like sleep latency (how long it takes to fall asleep) and sleep efficiency. It also has an adaptive alarm so that you can be woken up at the end of a sleep cycle instead of mid-dream. It's accuracy isn't perfect, but it's fairly good. (When I'm awake, it registers me as awake at least 90% of the time). Because Zeo only takes data in 2-minute increments, it cannot notice short awakenings. Hopefully, the next version will have an accelerometer or a bone conduction microphone to detect apnea, limb movements, and snoring.
In addition to the Zeo's data, I manually wrote down how happy, focused, and stressed I was each day, as well as a detailed 2-sentence description of what I did. From there, I derived several more quantitative measurements, like whether I exercised, if I slept alone, if I had sex etc. Then I dumped it all in a big spreadsheet and looked for correlations between measures of sleep quality and my activities. Several useful correlations popped out.
Here's a subset of what I found after gathering four months of data:
One issue that has always bugged me is sleep latency -- the amount of time it takes me to get to sleep. Here's what was correlated with lower sleep latency:
- Get exercise (though don't do it too late in the day). I had heavy exercise on 66% of the low latency nights vs 42% of the high latency nights.
- Going to bed earlier. This is counterintuitive, as I'd expect myself to be more tired later in the night. (Average 1:16am bedtime for low latency nights vs, 2:04am for high latency) From personal experience, I know sleep latency gets much longer again if I go to bed before midnight. There's basically an ideal window.
- Don’t have intense new social connections in the evening I'm not about to make my life more boring to sleep better, but I can be more proactive about exercising or taking melatonin to help myself sleep. (10% of low latency nights had intense new social connections, whereas 38% of high latency nights did. The same applies to arguing at night.
- Don’t sleep alone Having someone sleep next to me helped me get to sleep faster. (52% of low latency nights were with a partner, while 28% of high latency nights with partner) Just make sure the bed is big enough!
- Have evening sex The effect was astonishing. (22% of low latency nights had evening sex, vs 0% of high latency nights) Even morning/afternoon sex has an impact (31% of low latency nights had morning/afternoon sex vs 22% of high latency nights)
Exercise:
Digging through the data more, I discovered other interesting things.
Exercise turns out to be good for a lot of things:
- Earlier to sleep (1:37 vs 1:52)
- Increased total sleep (436 vs 400 min (+36min))
- Reduced sleep latency (22 min vs 25 min (-3min))
- Increased REM sleep (153 vs 137 min (+16min))
- Happier, more engaged, less stress that day (6.9 vs 6.1, 6.5 vs 5.9, 3.3 vs 4.0 on 1-10 scale)
- Happiness/engagement/stress benefits even carry over to next day (6.6 vs 6.3, 6.4 vs 6.1, 3.5 vs 3.8)
This is one of the things that encouraged me to start getting daily exercise.
----
Correlation vs causation:
As any good scientist knows, correlation does not imply causation. In regular terms, if two things tend to happen together, it doesn't mean one directly causes the other. There could be a third thing that causes both of them. Thus, doing one may not cause the other.
For example, the correlation between having a sleeping partner and shorter sleep latency could be explained for many reasons. When I have a sleeping partner, I'm more likely to have sex before bed, spend evenings out, cuddle before bed, and not work or browse the internet late at night. Any of those could be the actual cause. However, doing that kind of multivariate analysis requires a lot more data, as, for example, there are only a handful of nights where I had sex before bed but no sleeping partner. I could run a controlled experiment, but at some point I have to stop being a scientist and live my life. "No sex tonight honey, I need more data for the control group" is a great way to ruin a relationship. :-) In the end it's good enough to know that being in a healthy relationship helps me sleep better, regardless of the mechanism.
---
Application to other areas:
As I mentioned in the beginning. this technique of personal data mining does not just apply to sleep; it can be used against things like irritable bowel syndrome. I tracked what I ate every day for a month, and noted if I had gas or diarrhea. From there I could correlate the consumption of various foods with digestive issues. By avoiding those foods, I've substantially reduced instances of IBS.
---
Conclusions
Overall, by paying personal attention to numerous factors and making use of some self-tracking technologies, I was able to make lots of improvement to my quality of sleep. While medical advice and technology was useful, the ultimate evaluation of the effectiveness of everything was up to me. I had to be willing to self-track a variety of things, including things that had no known connection to sleep, and vary my life to include different behaviors. Most of the interventions I tried didn't work, and what worked for me may not work for you.
I think if these approaches are going to be adopted broadly, a few things need to happen:
- Patients' relationships with medical professionals need to change. Patients need to become active partners in solving health conditions. This means that interactions with medical professionals need to be longer and more frequent, perhaps with automatic sharing of health data between visits. Given the cost of medical degrees and the way doctor visits are billed, it likely won't be doctors that would be doing this. Insurance probably won't pay for it anyway.
- Data collection needs to be super easy. Techniques for automating data collection are great, and any manual data collection should be available via a wide variety of methods so that everyone has a method they find easy -- eg a phone app, text messaging, paper forms that can be scanned or photographed to extract the data, websites etc.
- Wikis and social networks need to make the sharing of potential remedies easy. Sites like CureTogether do a good job of giving people a space to share potential remedies. Voting systems help ensure that suggested remedies aren't totally off base, and these potential remedies are really just meant as raw material for experimentation so it doesn't matter if many of them don't work. With experimentation, you'll find the ones that work for you.
----
Introduction:
"Diffuse Conditions" is a term I made up to describe health conditions that have numerous potential causes and numerous potential treatments. These conditions include insomnia, allergies, irritable bowel syndrome, depression, and anxiety.
Diffuse conditions are hard to treat via the medical system. Due to the economic, research, and procedural constraints of the medical system, patients are often steered toward a narrow set of potential treatments. Moreover, these treatments are generally expensive drugs or procedures rather than simple practical advice. In addition, because of the prescription system, patients are only able to try one potential treatment per doctor visit, and since diffuse conditions have many potential cures, that potentially means a lot of doctor visits to find one that works. In addition, the diffuse conditions are sometimes bad enough to affect a patient's quality of life, but not bad enough to merit "medical attention". Doctors are generally focused on fixing people when they are broken as opposed to tuning them for optimal function. Basically, all this means that someone with a diffuse condition is likely to have a lot of difficulty finding a solution, and will end up spending a lot of money (either directly or via the tragedy of the commons system known as health insurance) trying to find a solution. At this point, healthcare is already up to 17% of our GDP in America, so we need to figure out a cheaper approach. I believe medical experts can be useful in treating diffuse conditions, but they need to empower patients to engage in a highly iterative cure discovery process.
The solution, I believe, is to data-mine your own life. This is a five-step process.
1. Get quantitative tools for measuring your performance.
2. Try various interventions.
3. Track performance and interventions over time
4. Look for correlations to suggest ways you can improve your condition
5. Share your findings online so that others can see if your successful interventions work for them.
-----
I'm going to provide a personal example of this to show how it can be done:
Sleep videos:
I've always had trouble with insomnia. For a long time, I just ignored it as a fact of life. In my early twenties, I decided to finally do something about it and get a sleep study. I went to one of the best sleep clinics in the country (Stanford). They found some minor issues (sleep apnea so mild that it could not be designated "mild sleep apnea") but nothing medically actionable. Unfortunately, that gave me little to act on. A sleep study requires you to sleep in a hospital with an unbelievably large amount of equipment attached to your head. Sleep studies are so expensive that any given person is only likely to do one or two of them. So unfortunately a sleep study is not a good way of determining anything useful.
I got the idea to use a cheap near-infrared video camera (Sony sells analog "nightshot" video cameras that can see in the dark) to do my own sleep study at home by doing time-lapse recordings of myself sleeping. I ended up with fascinating videos like these, which show the range of strange things that happen during the third of our lives when we aren't conscious.
Here's what I learned and did about it:
- I move around a lot while I sleep. Limb movements were documented on the sleep study. They appeared to correlate either with my posture (specifically, lying on my back) or with dreaming. The limb movements appeared to be impacting my quality of sleep as they appeared to often line up with or precede awakenings as seen on the sleep study. By looking at several night, I was able to see that the limb movements most often appeared when I was dreaming, regardless of posture. Also, as any good scientist knows, it's good to have a control group for comparison. As a result, I taped myself sleeping next to three different people, and found that I moved around substantially more than any of them did.
- If I'm sharing a bed, it needs to be king-sized if I am to sleep well. The video shows many instances of sleep-disturbance ping-pong, in which one person shifting causes the other person to shift, and that's in a king bed. In a smaller bed, it's even worse.
- If I'm sharing a bed, everyone needs their own quilt. Quilt-sharing dramatically worsens sleep disturbance ping-pong.
- Morning light dramatically lowers sleep quality. The morning light stimulates production of hormones to help me wake up, but this is not useful if I'm going to bed many hours after sunset. As a result, I started using heavy curtains to help myself get a full night's sleep.
- I don't have Restless Leg Syndrome. Based on the movements, I thought I might have it, so I got a doctor to prescribe the two most commonly used RLS medications. Both of them made my sleep worse.
- Posture interventions didn't change my quality of sleep. It appears that I need to spend some time sleeping on my left side, my right side, and my back. If I choose one particular posture to fall asleep in, I will spend most of the tail end of that night in the other two postures. If I force myself into a particular posture (eg by wearing a shirt and stuffing a pillow into the back), I fight it intensely. I think this may have to do with circulation; no part of my body likes to be compressed for long periods of time. I dislike extended sitting when I'm awake.
- Anti-apnea technologies don't help reduce the movements. CPAP is loud, messy, and obnoxious, and actually made the movements worse.
----
Sleeping pills:
I also did an evaluation of just about every type of sleeping pill I could get my hands on, and rated how effective they were at getting me to sleep and how much of a "hangover" effect they have the next day. I evaluated them via self reports, and sometimes with the videos, for how much they affected my sleep quality. Your results may vary. Here's what I found:
- Most over-the-counter sleeping pills (Tylenol PM, Nytol, Sominex, Benadryl) are bad. They tend to be based on the antihistamine Diphenhydramine HCl. While they are great at getting me to sleep, they worsen apnea (probably by relaxing the throat muscles too much) and they seem to make me stupider the next day. I was excited when earlier this year, a study came out showing that my "it makes you stupider" observation isn't just anecdotal.
- Alcohol (even a single drink) is a bad sleeping pill.
- Valerian was okay (decent effectiveness but some hangover), as was Lunesta.
- Melatonin was best for everyday use, and it's available over the counter. Your body produces it naturally, and you're probably already deficient in it if you are around bright lights before you go to sleep. I found 1mg was sufficient to do the trick. At >= 3 mg, I started to have some hangover effect the next day and some habituation issues (making it harder to go to bed the next night without it). However, at 1mg, I had very little hangover. I now tend to take it if I haven't been able to fall asleep after 45min.
- Ambien is useful as a "nuclear option" when I'm taking a transatlantic flight or on a packed overnight bus. It has some hangover, but it's shockingly effective. It has the bonus effect of helping reset your circadian rhythm to a new time zone if you take it at the bedtime you desire for the new time zone. Ambien has been known to have some substantial side effects, like sleepwalking, in certain people. In addition, before putting you to sleep it provides you with a feeling not unlike drinking an entire six-pack of beer in one sitting. I wouldn't be surprised if it eventually ended up as an illegal drug.
----
Quantitative interventions
While the above research yielded a lot of gains, I was able to learn even more when I started using automatic sleep-tracking technologies like the Zeo. Zeo uses an EEG headband to monitor your brainwaves. It can determine if you're awake, dreaming, in light sleep, or in deep sleep, by looking at them. From there, it comes up with derivative measures like sleep latency (how long it takes to fall asleep) and sleep efficiency. It also has an adaptive alarm so that you can be woken up at the end of a sleep cycle instead of mid-dream. It's accuracy isn't perfect, but it's fairly good. (When I'm awake, it registers me as awake at least 90% of the time). Because Zeo only takes data in 2-minute increments, it cannot notice short awakenings. Hopefully, the next version will have an accelerometer or a bone conduction microphone to detect apnea, limb movements, and snoring.
In addition to the Zeo's data, I manually wrote down how happy, focused, and stressed I was each day, as well as a detailed 2-sentence description of what I did. From there, I derived several more quantitative measurements, like whether I exercised, if I slept alone, if I had sex etc. Then I dumped it all in a big spreadsheet and looked for correlations between measures of sleep quality and my activities. Several useful correlations popped out.
Here's a subset of what I found after gathering four months of data:
One issue that has always bugged me is sleep latency -- the amount of time it takes me to get to sleep. Here's what was correlated with lower sleep latency:
- Get exercise (though don't do it too late in the day). I had heavy exercise on 66% of the low latency nights vs 42% of the high latency nights.
- Going to bed earlier. This is counterintuitive, as I'd expect myself to be more tired later in the night. (Average 1:16am bedtime for low latency nights vs, 2:04am for high latency) From personal experience, I know sleep latency gets much longer again if I go to bed before midnight. There's basically an ideal window.
- Don’t have intense new social connections in the evening I'm not about to make my life more boring to sleep better, but I can be more proactive about exercising or taking melatonin to help myself sleep. (10% of low latency nights had intense new social connections, whereas 38% of high latency nights did. The same applies to arguing at night.
- Don’t sleep alone Having someone sleep next to me helped me get to sleep faster. (52% of low latency nights were with a partner, while 28% of high latency nights with partner) Just make sure the bed is big enough!
- Have evening sex The effect was astonishing. (22% of low latency nights had evening sex, vs 0% of high latency nights) Even morning/afternoon sex has an impact (31% of low latency nights had morning/afternoon sex vs 22% of high latency nights)
Exercise:
Digging through the data more, I discovered other interesting things.
Exercise turns out to be good for a lot of things:
- Earlier to sleep (1:37 vs 1:52)
- Increased total sleep (436 vs 400 min (+36min))
- Reduced sleep latency (22 min vs 25 min (-3min))
- Increased REM sleep (153 vs 137 min (+16min))
- Happier, more engaged, less stress that day (6.9 vs 6.1, 6.5 vs 5.9, 3.3 vs 4.0 on 1-10 scale)
- Happiness/engagement/stress benefits even carry over to next day (6.6 vs 6.3, 6.4 vs 6.1, 3.5 vs 3.8)
This is one of the things that encouraged me to start getting daily exercise.
----
Correlation vs causation:
As any good scientist knows, correlation does not imply causation. In regular terms, if two things tend to happen together, it doesn't mean one directly causes the other. There could be a third thing that causes both of them. Thus, doing one may not cause the other.
For example, the correlation between having a sleeping partner and shorter sleep latency could be explained for many reasons. When I have a sleeping partner, I'm more likely to have sex before bed, spend evenings out, cuddle before bed, and not work or browse the internet late at night. Any of those could be the actual cause. However, doing that kind of multivariate analysis requires a lot more data, as, for example, there are only a handful of nights where I had sex before bed but no sleeping partner. I could run a controlled experiment, but at some point I have to stop being a scientist and live my life. "No sex tonight honey, I need more data for the control group" is a great way to ruin a relationship. :-) In the end it's good enough to know that being in a healthy relationship helps me sleep better, regardless of the mechanism.
---
Application to other areas:
As I mentioned in the beginning. this technique of personal data mining does not just apply to sleep; it can be used against things like irritable bowel syndrome. I tracked what I ate every day for a month, and noted if I had gas or diarrhea. From there I could correlate the consumption of various foods with digestive issues. By avoiding those foods, I've substantially reduced instances of IBS.
---
Conclusions
Overall, by paying personal attention to numerous factors and making use of some self-tracking technologies, I was able to make lots of improvement to my quality of sleep. While medical advice and technology was useful, the ultimate evaluation of the effectiveness of everything was up to me. I had to be willing to self-track a variety of things, including things that had no known connection to sleep, and vary my life to include different behaviors. Most of the interventions I tried didn't work, and what worked for me may not work for you.
I think if these approaches are going to be adopted broadly, a few things need to happen:
- Patients' relationships with medical professionals need to change. Patients need to become active partners in solving health conditions. This means that interactions with medical professionals need to be longer and more frequent, perhaps with automatic sharing of health data between visits. Given the cost of medical degrees and the way doctor visits are billed, it likely won't be doctors that would be doing this. Insurance probably won't pay for it anyway.
- Data collection needs to be super easy. Techniques for automating data collection are great, and any manual data collection should be available via a wide variety of methods so that everyone has a method they find easy -- eg a phone app, text messaging, paper forms that can be scanned or photographed to extract the data, websites etc.
- Wikis and social networks need to make the sharing of potential remedies easy. Sites like CureTogether do a good job of giving people a space to share potential remedies. Voting systems help ensure that suggested remedies aren't totally off base, and these potential remedies are really just meant as raw material for experimentation so it doesn't matter if many of them don't work. With experimentation, you'll find the ones that work for you.
