Monday, July 27, 2020

The Curve is Starting to Flatten Again

Welcome back to our coronavirus series. Here is a quick summary of my earlier posts:

-Because the growth rate is exponential, it's best to take logs of the data
-I prefer data on active cases rather than total confirmed cases. This is because people who have recovered or died aren't spreading the disease anymore. Unfortunately, I haven't found good data on recoveries. That is why I look at the total number of confirmed cases.
-I run statistical tests to find significant changes in the trend
-The curve was flattening for a while until late June

The tests found 7 distinct periods. The first period was the beginning until March 22. Growth was very rapid back then. You can see that in the table below; the coefficients are circled. The number next to "t1" is the slope in the first period. The slope drops from 0.14 to 0.12 for the second period (March 23-April 13). It keeps falling but then it begins to rise in period 5, which began on June 17. It rises again on June 27. But now we are in period 7. It started on July 18 and the slope began to decline again. The number of cases is still going up, but at a slower rate now.

I graphed the data along with the trend lines, but it is hard to see all of them in the top graph. In the bottom graph, I only show the trends.

Stay safe!

Saturday, June 27, 2020

Coronavirus - Statistics Update

We have been warned that as America reopens, the virus will spread more rapidly. But this didn't show up in the data until very recently. About a week ago, I did a quick check and didn't see any evidence. It's different now.

My data comes from Bing's COVID tracker. I prefer to focus on active cases, since you can't be infected by people who have recovered. However, I noted earlier that the recovery data is not very reliable. First, not every county's recovery data is reflected in Bing's tracker. The data that is reported appears to be cumulative recoveries rather than daily recoveries. That's fine, but on some days, the cumulative total of recoveries goes down - which should be impossible. It can't be daily recoveries because the sum of them all eclipses the total number of cases - also impossible.

Instead, I focus on total confirmed cases. Since the virus spreads exponentially, it's best to take logs (see my explanation in an earlier post). In the graph below, "logC" is the log of confirmed cases and "t" is time. The rapid growth slowed around late March and early April. It continued to slow for a while until the very end of the graph. It's hard to see, but that's why we rely upon statistical tests rather than just eyeballing pictures.

The tests found that there were 6 different periods. In each period, the slope is significantly different from the previous period. For the first 5 periods, the slopes were getting closer and closer to 0. That's a good thing. It means that the disease's spread is slowing down. Unfortunately, Period 6 (June 21 - present) is different. The slopes for Periods 5 and 6 are circled in the picture below.

The slope is about the same as it was in Period 4, which was mid May. It's true that testing has expanded. This means that we are detecting more cases than before. However, it does not entirely explain the increase. The expansion in testing started well before June 21. I don't know if new lockdowns are justified. The benefits of flattening the curve have to be weighed against the economic costs. I'm working on a research project to address this, but progress has stalled. In the meantime, I hope you're staying safe and healthy.

Monday, June 1, 2020

America is Reopening, but the Curve is Still Flattening

As the shutdowns gradually end, the big fear is that there will be a surge of new cases. I find that the number of active cases has continued to rise. However, the growth rate has not increased. The curve is still flattening.

My data comes from Bing's Covid-19 tracker. I focus on active cases (active = confirmed - recovered - deaths). Neither the dead nor the recovered can spread the virus - that's why active cases are the relevant factor. My earlier blog post was about total confirmed cases, but when I turn my attention to active cases, I still get the same result: the curve started to flatten in early April.

Here is the graph of active cases in the US.

You can see the exponential growth. As I explained earlier, when you have exponential growth, it's better to look at the log graph:

It's still spreading, but not as rapidly as before.

When examining the data, I asked, "When did the curve's slope change?" I found that it changed several times.

Time1 is before March 20. Time2 is from March 20 - April 7. Though this was during the lockdown, the disease was actually spreading more rapidly than before. However, it can take up to 2 weeks before symptoms appear, so this increase reflected cases that had started earlier. Time3 is April 8 - April 29. Here we see the curve flattening: the coefficients (circled in red) started shrinking. However, they were still positive. Overall, that means that the virus was spreading but the growth rate was slowing down. Time4 is April 30 - May 9. Some states had started reopening by then. Nevertheless, the growth rate kept slowing down. Right now, we are in Time5: May 10 - present. The growth rate is even slower.

This analysis does *not* prove that the reopenings were a success or that shutdowns were unnecessary. A longer shutdown would have driven down the growth rates more quickly, but we don't know if that benefit outweighs the economic costs. That question is too complicated for a single blog post. I have one piece of the puzzle: the curve is still flattening in spite of the reopenings. Other researchers will have to fill in the rest.

Friday, May 15, 2020

K+Q vs. K+R: How to Win This Tricky Ending

Click here if the diagrams aren't displaying correctly

Over the last few weeks, I have been exploring ChessTempo's Endgame Benchmark problems. Most of them weren't too hard and my rating rose steadily. But then I ran into a bunch of Queen vs. Rook exercises. They were extremely difficult. I don't think most players realize how tough they are. Successfully solving them would often take more than an hour plus a bit of "luck." (I.e., the computer playing a move that delays checkmate but is easy to refute). In a practical game, this could be even harder. The defensive techniques are very simple, you won't have hours and hours of time on the clock, and you'll be exhausted from the long battle that preceded the endgame. So here is my guide to winning this tricky ending.

Tip #1: *Always* watch out for stalemate

I was working on a problem, and the following position came up in my calculations. You can quickly verify that there is no safe square for the rook. Either it falls immediately or it falls a few moves later due to tactics. I was about to head for this position, but then I remembered to look for stalemate.

[Event "?"] [Site "?"] [Date "????.??.??"] [Round "?"] [White "New game"] [Black "?"] [Result "*"] [SetUp "1"] [FEN "8/8/1Q6/8/k7/8/1K6/2r5 b - - 0 1"] [PlyCount "1"] 1... Rc5 $1 {and the rook is immune. White's pieces don't coordinate very well and the win will be difficult.} *

The next position is well known.

[Event "?"] [Site "?"] [Date "????.??.??"] [Round "?"] [White "New game"] [Black "?"] [Result "*"] [SetUp "1"] [FEN "5k2/7r/4Q1K1/8/8/8/8/8 b - - 0 1"] [PlyCount "3"] [TimeControl "40/7200:20/3600:1800"] 1... Rg7+ $1 (1... Rh6+ $1 {also works}) 2. Kf5 (2. Kh6 Rh7+ $1) (2. Kf6 Rg6+ $1 {This idea is especially important to know. It comes up in a lot of variations}) 2... Rf7+ {The only way to escape the checks is to play 3.Ke5 or 3.Ke4. But in either case, Black draws immediately with 3...Re7.} *

Tip #2: Centralize the queen

We all know that a centralized queen controls more squares than a cornered queen. Thus, if you keep your queen in the center, she can often pick off the rook with a fork. So far I'm just stating the obvious. But if you combine the first 2 tips, you may learn something new: when you push back the defender, lead with your king, not your queen. To see this, change the last diagram just a little.

[Event "?"] [Site "?"] [Date "????.??.??"] [Round "?"] [White "New game"] [Black "?"] [Result "*"] [SetUp "1"] [FEN "5k2/7r/6K1/3Q4/8/8/8/8 b - - 0 1"] [PlyCount "4"] {The situation is completely different.} 1... Rg7+ (1... Rh6+ {doesn't work any more. Because White kept his queen in the center, there are no stalemates.} ) (1... Re7 {(to prevent 2.Qd8#)} 2. Qd8+ Re8 3. Qf6+) 2. Kf6 Re7 ({ Another important difference:} 2... Rg6+ {isn't stalemate. Again, there are no stalemates because the queen stayed in the center.}) 3. Qd6 {and the rook falls } *

Tip #3: The Temporary Retreat

At first, it's easy to push the defender back. The tough part is once they are near the edge, it's hard to make further progress due to stalemate threats. The defender should keep the rook on the third rank. This simple idea will probably frustrate the attacker unless he knows about the Temporary Retreat.

[Event "?"] [Site "?"] [Date "????.??.??"] [Round "?"] [White "New game"] [Black "?"] [Result "*"] [SetUp "1"] [FEN "8/8/8/8/3Q4/1k1K4/8/2r5 w - - 0 1"] [PlyCount "3"] [TimeControl "?"] 1. Qe5 $1 (1. Qb6+ {is tempting, since it pushes back the enemy king. But White's king cannot approach and there's no way to fork the rook.}) 1... Rd1+ 2. Ke2 {At first glance, White is losing ground. But Black's blockade quickly collapses. The rook can't hold the d-file because of forks (I'll let you work out the details). Trying to give checks from the side with 2...Rg1 or 2...Rh1 fails for the same reason. Returning to the c-file doesn't work either: 2... Rc1 3.Kd2 and the checks end shortly. The rest is straightforward.} *

Thus, the Temporary Retreat is very effective. However, it's so counterintuitive that you probably wouldn't find it during a game. Voluntarily giving up ground looks is very unnatural; almost everyone will prefer 1.Qb6+ instead and then struggle to break the blockade. Now there is one more paradoxical idea to know, which brings us to Tip #4.

Tip #4: Letting the King escape

[Event "?"] [Site "?"] [Date "????.??.??"] [Round "?"] [White "New game"] [Black "?"] [Result "*"] [SetUp "1"] [FEN "8/8/8/4Q3/8/k2K4/1r6/8 w - - 0 1"] [PlyCount "25"] 1. Qa5+ {My first instinct was to approach with the king, but then I just get checked. So instead, I intentionally let Black's king escape from the a-file.} Kb3 2. Qa1 $3 {And now I violate Tip #2 and put my queen in the corner. At first, White's last 2 moves look terrible. But let's look deeper. While Black's king did escape from the a-file, he can't run any further - he's chained to the rook. Furthermore, the Black king is taking away squares from the rook. Black's pieces are badly misplaced and he's in zugzwang.} Rh2 (2... Ra2 3. Qc3+ Ka4 4. Kc4 {and it's very easy}) 3. Qc3+ Ka2 (3... Ka4 4. Qd4+ $1 { (centralization) and the rook drops off shortly:} Kb3 { (other moves lose the rook even faster. I'll let you verify)} 5. Qd5+ $1 Ka4 { (again, other moves lose the rook more quickly)} 6. Qa8+ $1 { followed by 7.Qb8+ and 8.Qxh2}) 4. Qe5 $1 {Centralizing the queen. It's hard to believe, but White can't fork the rook yet. Instead, I take away squares and prepare to advance my king.} Rb2 (4... Rh3+ 5. Kc2 { and the centralized queen denies Black ...Rh2+}) 5. Kc3 Kb1 { The rook can't move because of tactics} 6. Qe1+ Ka2 7. Qd1 $1 ({ Remember our earlier principle: lead with the king, not the queen.} 7. Qc1 $6 Rb3+ {and now White can't play} 8. Kc2 $4 {because of} Rc3+ $1 {. So White has to play 8.Kc4 and then he's driven back even further after 8...Rb4+!}) 7... Rh2 {Black is in zugzwang; all the rook moves lose quickly.} 8. Qa4+ Kb1 9. Qb5+ Ka2 10. Qa6+ Kb1 11. Qb6+ Ka2 12. Qa7+ Kb1 13. Qb8+ { winning the rook and the game} *

And that is how you win Queen vs. Rook. Stay healthy everyone!

Tuesday, April 14, 2020

The Curve is Almost Certainly Flattening

Almost all chess events have been cancelled due to the coronavirus, so instead we use our statistical tools to analyze the epidemic. To flatten the curve, the US has shut down much of the economy. Are these efforts succeeding? At first glance, it looks like we haven't made any progress.

Confirmed cases in the US

(Source: CDC)

However, the trend is exponential, so it can be hard to spot changes in the growth rate. That's why we take logarithms:

Log of confirmed cases in the US

Now there does appear to be a recent slowdown in the growth rate. I checked this with a simple model. New York State announced a lockdown on 3/20. Within a few days, other states with big outbreaks made similar announcements. It can take a week or two for an infected person to show symptoms. Thus, even if the social distancing is working, you might not see results until the end of March or early April. So does the curve flatten starting in late March/early April? Yes, it does. In the picture below, "time73" is for the curve after the first week of April. "Time" is for the whole curve. You can see that the number next to "time73" is substantially smaller - this means that the curve started to flatten in early April.

The next part of this post is more technical. If you haven't studied a lot of stats, this is a good time to stop reading. The bottom part of the screenshot shows that the slowdown in the growth rate is statistically significant. However, this test presumes that we had already known the exact date of the structural break. We didn't. It could have been anywhere from 1 week to 2 weeks after the lockdown.  Andrews (1993) showed how to test for an unknown structural break (see Table 1 in this paper). However, the slowdown remains statistically significant after making these adjustments. Thus, I conclude that the curve really is flattening.

Though I looked for a structural break based on when the social distancing started, the test does not prove that the lockdown caused the curve to flatten. It seems likely to be true, but showing that is more complicated. I guess I would start by comparing the results for different states that began the lockdown at different times (while being aware of the endogeneity). I don't have time to explore that, but hopefully one of my readers does. Write a comment with your results. My analysis also does not show if the shutdown was worth it. That would be a separate project. 

That's all for today. Stay safe, everyone!

Sunday, March 15, 2020

Candidates Tournament

Caruana is the favorite, but the opposition is very strong - don't be surprised if someone else wins.

Monday, February 10, 2020