Monday, November 4, 2013

Hurricane Sandy, One Year Later

It's been a year since Hurricane Sandy hit the east coast, and although it's not talked about as much in the national media, things are still not back to normal.  Being born and raised in New Jersey and now living half a country away in Texas, I wanted to look into how bad things still are.  I visited the beaches of southern New Jersey last summer and everything looked pretty much the way I remembered it, but I know that other areas were much harder hit.

When I was home I was bombarded with these "stronger than the storm" commercials, and I thought it was strange that they seemed to only focus on the Jersey shore's recovery.  I got the impression from people I talked to that the majority of the recovery effort has been focused on New Jersey's moneymaker: its beaches.  While this at first made sense to me, the longer I thought about it the more unfair it seemed.  Many of the shore homes being rebuilt are people's second homes--a luxury item.  Sure it sucks to lose that shore house, but it sucks a lot more to lose your only house.  Anyone in that situation would feel pretty left out of the recovery, especially given how much attention is focused on rebuilding the shore.

I set out to visualize the damage from Sandy as well as the recovery.  I wanted to know if there was an income gap in the recovery, and whether there was a bias towards rebuilding the shore at the expense of inland areas.  It turns out, this was harder than I thought.

I started with housing assistance data from FEMA.  Here they break the total housing damage down by zip code and also list the amount of FEMA aid given.  Datasets are split among homeowners and renters.  Since the total damage is not listed for renters, I decided to base my analysis on the homeowners data.  This will skew my results towards higher-income families though, since most low-income families rent.

I wrote a Python script to parse this file and extract some values for analysis.  I initially set out to make maps by writing directly to a blank image of the state with its zip codes identified.  To do this, I used the Beautiful Soup package to parse the XML in a similar way to what I did in my first post.  However, I soon found that this was ugly and uninformative.  I needed to overplot my results onto a real map with cities listed so that people could identify where the damage is.  I looked all over for a Pythonic way to do this, but I never found anything as good as ggmap in R.  The only problem is that I don't know R very well.  My solution: do everything in Python and only write out what needs to be plotted in R!  I did the analysis, set the scale for the fill color, and created a mapping from zip code to fill color all in Python, then just dumped this out to be read by my R script.

Here's the first map, a map of total homeowner damage in the state.

source: Python R
As expected, the worst of the damage is near the coastal regions.  Also, the damage at the beaches north of Atlantic City was much worse than at the southern beaches I visited this summer.  That part makes sense too.

Now here's a plot of total damage minus the amount of FEMA housing aid.

source: Python R

That plot doesn't look too much different from the first.  The color overlay is lighter in South Jersey, indicating that much of the area has at least received the money to cover rebuilding costs.  Again, when I visited my parents near Philadelphia, that was pretty much the picture that I saw.

I also found data from the office of the Comptroller, compiling all the Sandy-related contracts awarded for recovery.  I can sum these up and subtract from the total damage to get another estimate for how recovery is going.

source: Python R

Again, it's the coast that still has the largest deficit between recovery assistance authorized and damage done.  Clearly these data are telling me that my initial understanding of the situation was wrong.  The state is right to spend more money on rebuilding the shore because that's where most of the damage was.

However, there's a flaw in this analysis.  I'm still only analyzing the housing damage reported by homeowners.  I'm missing a large group of the state's population, and probably those with the lowest income who were financially hit the hardest.  I've tried to account for this by scaling the damage by the mean income for each zip code in the FEMA claims, but this number is self-reported and not properly normalized by household.  The resulting maps didn't look much like anything so I dropped it.  In the end, I guess I learned that coastal storms do more damage to coastal populations.  Wow.  What a breakthrough.

4 comments:

  1. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. Johnny J - been reading your blogs since you started! Great post again! Very interesting topic, and I agree, everything looked good when I was in NJ last. However, I was only near Philly and near the coast...

      Makes sense that most dollars are spent on the shoreline - especially in the comptroller numbers since some funds are spent on debris cleanup, corps of engineers, other costly, inefficient etc expenses..

      I had a couple thoughts looking for a possible income gap in the recovery, and whether there was a bias towards rebuilding the shore at the expense of inland areas:


      1) If you're correct about higher income being skewed towards vacation homes and the shoreline, wouldn't private insurance & folks paying for their own damages cover the majority of issues? Thus making the number of dollars spent on the shoreline even more significant towards your case? I would imagine anything the shore spends that is higher between the FEMA and Office of the Comptroller would support your case (disclosure: I'm not an expert on flood insurance).

      2) It would be great if you could measure population decline by city, similar to post-Katrina in New Orleans. However, I don't think that's possible to obtain that data in such short intervals.

      3) It would be extremely interesting to see the economic changes by city, or by county based on income level. Comparing last summer and this summer tax receipts should get you to this, which should be easily obtainable. I'd imagine a big change in tax receipts is a good indicator of how a county is doing year over year. I'd estimate that all counties were hit relatively hard, however over time you can tease out who is recovering faster (albeit with a very limited amount of months). You'd have to normalize for improving economic conditions across the country right now.

      4) To add to #3, a good indicator of how everything is going can be to look at change in government deficit compared to cities or counties- NJ was widely underfunded and ran a large deficit prior to the hurricane, but it would be interesting to look at changes across counties since and also compare with cities with major population declines such as New Orleans or Detroit (or CA as a state) as an indicator to more trouble along the way. There would obviously be some confounding variables in the way here as well though.

      Delete
    2. Hey Garrett,

      Thanks for the really detailed comments! I'm glad you've been reading.

      1) That's a good point. I was using damage reported on FEMA claims as a proxy for total damage, but I hadn't realized that if private insurance or other means could pay for all your damages, you're not going to file a FEMA claim.

      2) Yeah, that would be interesting to see. I don't know if there would be a comprehensive population survey between census periods though.

      3) That's a great idea. When I get some more free time I'm going to poke around with this. I did a little googling and all I could find were statewide summaries of tax receipts. I would think this should be available by county, but I haven't found it yet.

      4) I think it would be difficult to subtract off all the competing trends to isolate an effect from the storm. What you suggested for #3 is probably the cleanest way to get at economic impact. Using the deficits run by counties will have to account for changes in funding at the state level and probably nationally.

      Delete
  2. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. IEEE final year projects on machine learning In case you will succeed, you have to begin building machine learning projects in the near future.

    Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.


    Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.

    ReplyDelete