Tag: data

The difference between the US Debt & Deficit.

The super short version:

Let’s say Sam gets $50 per year in allowance (revenue), but owes his Mom $100 in “debt” after two years.  He accrued that debt by spending $100 per year. So, he spent $50 more per year  (“yearly deficit”) than he made in allowance.

2 years * $50 yearly deficit = $100 debt.

 

As told by others:

More depth of the difference between debt and deficit: http://ptmoney.com/us-federal-debt-vs-deficit/

Even greater detail specifc to the US economy as it relates to taxes, social security, military spending etc:

http://useconomy.about.com/od/fiscalpolicy/p/US_Debt_Deficit.htm

The History of the US Debt

Breakdown of US Debt Over Time:

Here is a breakdown of debt accrued over 40 years:

http://www.skymachines.com/US-National-Debt-Per-Capita-Percent-of-GDP-and-by-Presidental-Term.htm

Here is a picture snapshot from the site incase your feeling lazy:

 

 

 

 

 

 

 

 

 

 

Per Presidency Comparison:

This is the total debt accrued per president in a more condensed format:

 

 

 

 

 

 

 

 

I crossed checked it with the treasury data to make sure it was kosher – you can too: http://www.treasurydirect.gov/govt/reports/pd/histdebt/histdebt.htm

 

Reflection:

The great thing abot history and numbers is it help remind us there is a difference between what rememebr and what really hapened.

If your going to base your opinions on these numbers and term, please know what they are, or else not only are you doing the country a diservice, but you are letting people use your naivety to control your feelings on an issue.

Opinion:

As you can see it isnt a question of president,  the larger issue is that spending more than we make has been an issue without our country as a whole for a long time. Many of us complain about the economy in a warm room with a bed to sleep onand food in our stomach. (Of course many do not, and we need to fix that too), but we spend more than we should, and then hope someone will save us so we can do it again. That might be a better issue to reflect on then the unsubstantiated blame game.

Not the Facts Just the Data: More Google Census Data

I am posting some updates to census stats since, as I said before, the data is there, it’s public and people should be exposed to it without ads or “interpretations” in the middle.

So, here you go:

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

There is no better quote to reference then your own, made from the source:Not the facts, just the data

Here is another post in my series of “Not the facts, just the data”. This post came after an argument I had with someone I care about. Make no mistake that this is not a rebutle but just a perspective on data vs. facts, and arguments vs. politics.

The biggest argument I stand being, from my point of view, is this: Do not trust or love politics, love your people, peers, country, and family.

Politicians on either side of the coin will use data for their agenda. If you get wrapped up in it, and start believing what you are *told*, through sound bites and ads, you will be misinformed 100% of the time. More information isn’t tantamount to a depth of information.

This is why I have learned not to trust politics, they bet on people not reading the full story (as wel only have so much time in the day) and try to create anger and conflict to make people mad enough to vote how they want. It feels like THAT is what has gotten worst over the years, not so much what is happening in the world, just the way its agenda is given to us. China does it by cutting off information, the US does it by overloading its people with one sided information.
The good news is that with the advent of the Internet it is now so easy to get the data and documents from the source, and fight back as a citizen saying “don’t use me for your agenda”. No need to go to the library and send letters requesting data that may take days or weeks; it is available now if you are willing to read it.
As my methods to get to the data evolve, I try and follow some procedures to navigate past the political propaganda.
First, If I find a “fact” that I am researching on any news site, I look for the citation for the “fact” so I know where it came from, and then look that fact up from it source to read it myself. If there is not citation, I start over. A citation does not make it data, it just helps you get a path to the source. Secondly, if I find the source I read it, if I cannot find the source I look for the source directly. At this stage in the evolution of my process I have deemed .gov sites and non-profit census bureaus as a final source. In some situation you can never be sure unless you count the numbers yourself, but I have found the most consistency at that level. Yes I know that even the government stats, non profits, and census bureaus can lie but if I can find some consitancy at that level I am pleased.
Lastly, how to spot a fact with skeletons in its closet that should trigger an investigation like the one above before having the debate.
Well, the main thing that triggers a fact check to me are statistics, and references to bill being voted on. Why? Well, over the years I have found that stats have a high potential to be screwed. Or example, what constitutes “1 year” – Fiscal year or calendar year etc. Another example is is the stats extrapolated from previous data or data that has already happened. Another read flag is the use of tag lines,  sound bits and general words like “taxes”, and “everyone”, and “higher” — higher then when and what for whom? Also, the reference to bills, are tough to believe from something like “he voted to….” After living in DC and working on capitol hill with you realize that both sides of the fence will quote the same document at different parts and come up with completely different conclusion. It is impossible for both are either to be entirely right, but both sides would never realize that they both *think* they are 100% right. As my readers and friends know by now, one of my big rules of thumb is: If experts disagree it is time to make your own decisions. A bill is written by many people with many agendas for their state. They are stiuffed, written and re-written over and over to try to get as much as they each want to get in it, and keep out as much as they don’t. They do this hours and hours everyday to come to a final document that is then voted on by all parties. It is very important to remember that time line, for no document has one message, and if it passed their is a big chance that both parties voted yes on it (unless there was a big split with few deciding votes). Point being, much could have gone into a document and so much more history to that document then any sound bite could articulate.
So, one of the bills that was brought as a point of contention was based on a sound bites saying “Obama voted on late term abortion”, and it was said that the bill was made to let mothers have the right to kill a baby after it is born alive. Well, personal, I did not know about that bill first hand, and that I promised to look it up. Speaking from my own perspectives I don’t like the sound of it, but I want the source and document first before my personal conclusions on that vote are spoken.
This is one of the actual bills voted on for Illinois referenced. It only took a few minutes to bring up. It was indeed named “Partial-birth Abortion Ban Act”, so that is a true data point.  As it turns out was voted on 44-7 (almost everyone agreed in both parties, that decided to attend). Also there were 5 present votes. What does a present vote mean?
 The “present” vote is in effect a “no” vote, but it is a “no” vote that sends a message. The “present” vote is used by lawmakers in situations where they agree with a bill in
spirit, however the current version of the bill is not good enough to vote “yes;”
The bill, in my first hand summary, would be this:  A bill for late term abortion *if* it will kill the mother to have the baby, but still says it is illegal otherwise. But please, do not take my word for it! Read it here:
So it is important for me to see just how complex a bill can be, in addition to how complex it is to create, finalize,vote on and pass, on it.
Another topic was immigration. Stats are the hardest data to really find concrete numbers. There isn’t a list of people that are counted ever. It is def not as easy to get to the bottom of as a document being references. Old census bureus that have been around and referenced over time by both parties is, to me, a better place to start than from a news site, network, or politician.
So here is census data from the Pew Foundation on immigration stats, highest in 2007, and leveled off down by a million the following years.
It is important to note that data usually doesn’t come with a “should”, “good”, “bad”, or “won’t” list of words around it. Why did it go up? That is as complex as why did it go down? And as you will see that once why is added to statistics rarely does a group of experts agree. There are so many variables, and that is where debates and politics I suppose have a place. But I would implore you all out there to start the debate after the data is presented, and be sure not to find yourself repeating an ad or politicians claim that a statistic is “up” or “down” until you read it for yourself from the source.
I think my new strategy will be to make sure that before I argue on big issues like this, especially in DC where the ads and politics are so ripe with contention, I will just say show me the document we are arguing over before getting involved. If we do, maybe the citizens of this country can take back their right to be informed, and to make decisions without political agenda being the force that drives them.

Gas: Remember. Learn. Be Smart.

Complaining about gas prices is like complaining that your love of ice cream has made you fat. This is the data points that should matter to you: http://cl.ly/0H0R1H312e1h0n1h3x42… Stop buying big cars that suck gas, EVEN when this fall/winter gas prices go down, as they usually do.

If you feel like a baller and you want to get a big car that sucks gas, then keep that top-dog mentality  and pay the high gas prices in with some pride. As a matter of fact, it would make more sense if you bragged about how much you get to pay for your big high-roller rich-boy status.

Again, please read the data before you shout form a hill and light those torches in the street, based on some guy in a suite trying to get the most sought after job in the world, and tells you something that makes him look good — no matter what side that may be.

As usual, here is my non-political mouthed stats and data, from the source, with links to where you can read more in case you are without time, or just lazy.:

http://www.opec.org/opec_web/en/about_us/25.htm

” (OPEC) was founded in Baghdad, Iraq, with the signing of an agreement in September 1960 by five countries namely Islamic Republic of Iran, Iraq, Kuwait, Saudi Arabia and Venezuela. They were to become the Founder Members of the Organization.

These countries were later joined by Qatar (1961), Indonesia (1962), Libya (1962), the United Arab Emirates (1967), Algeria (1969), Nigeria (1971), Ecuador (1973), Gabon (1975) and Angola (2007).”

http://en.wikipedia.org/wiki/OPEC

” OPEC members collectively hold 79% of world crude oil reserves and 44% of the world’s crude oil production, affording them considerable control over the global marke”

 

You also might have forgotten the highest increase in gas was between 2001 and 2008. Remember when we had all those awesome Hummers that were sold only a few years later? Oh and those wars in those countries that are the leaders of OPEC because we were “so angry”? Yeah – remember, learn, be smart.

Get more data about energy use and projections here: http://www.eia.gov/

 

Control what you can control, and think ahead next time some one out there tries to save up money in our national bank, asks you to help them cut down on our dependency on oil, and says you will look cool in a big gas thirsty car. Whether gas prices or high OR LOW.

 

Let’s have cleaner debates with our neighbors: Not the facts, just the data.

I am not trying to get entrenched in the political back-and-forth going on. I actually do understand all of the he-said she-said going back-and-forth when it comes to peoples opinions. Opinions are each persons right to have, especially when it comes to social philosophy. Additionally, opinions are hard to “verify”; your beleifs are your prerogative.

However, it is amazing to me how non-opinion based information gets thrown AS opinion. It is even more distressing because it is so easy to find many statistic directly from the source, before they are muddied by political agenda, or distributed in off-the-cuff comments and hearsay.

So, I figured I could help…

The following are just the data & graphs of screen-shots taken from real census data (and yes the URL of where I got the data from is also noted next to each graph.) Feel free to browse the data yourself and make your own observations.

Debate the implications all your want – but below are not news reports or debate notes, they are charts taken from the actual data sources. All I ask, and hope to acheive, is that no matter what side your on, just remember not to include non-sensicle bullet points that just aren’t true and instead try to argue around your beliefs. Do not get caught up in baseless, inflated, skewed, or inaccurate depictions of history as it relates to hard cold metrics.

Important notes on data around first day of office dates

The first day in office for elected presidents is in late January following the election results, coming two months before in November.

George W’s First Day Obama’s First Day
 January 20, 2001  January 20, 2009

The stock market
The stock market shows the amount of money distributed in US corporation. The rich, and anyone investing in the rich wins when this graph goes up. Note 2007 and 2008 were shockingly bad, the worst downswing since the mid 80s. The good news everyone is it has only taken four years to get back from our bubble burst of 2001 which took eight.
http://www.google.com/finance?q=INDEXDJX:.DJI (You want this going up)

 

Unemployment Rate

This one is a often heavy argument point, but the data is all very easy to lookup. When a single percentage is called out it is hard to know its context. Those that scream and point fingers from he top of a mountain often end up implicating everyone – if those at the bottom of the mountain did a little research…. Again 2007 was the beginning of the frenzy. And if you remember there were many sad days and foreclosures making an “america is shutting down” environment. It was the intense swing from just a year before in 2006 that scared people the most. 
http://goo.gl/2O2zl (You want this going down)

  2007-2009 was the sharpest upswing of unemployment we have seen over 30 years.

Debt

Here is another amazing story of how politicians program you to just repeat what they shout, programming you to have their debate. Again, the data is there to have your own debates and ones that aren’ skewed by anyone. Debt sucks, and gus, we have had it growing for a long time. And this rate back-and-forth argument is again a graph away. You can see the linear growth of date below.
http://goo.gl/d9Wvz (You want this going down)

 

Personal Income

This is one that people don’t talk much about, because we like focusing on the negatives. BUT your incomes have been growing fairly steadily for a long time. We had a short blip of our first downward trajectory in 2008, actually the only one in recorded history of the US, but it corrected itself pretty quickly after 2009.
http://goo.gl/2CaIc (You want this going up)

 

 

 


I also found interesting that the following census data shows a steady increase in revenue and income in the US for decades.

Disposable Income per capita

This is also a great graph to be going up. Like the income graph we saw our first real blip ever in 2008, but for the most part we have consistently been getting a linear growth.
http://goo.gl/QQOlG

Revenue

This one is a bit less cheerful, but t’s the reality. We had some great years as a country with revenue, but unfortunately the money stopped coming in in 2007 and we have all felt this graph at home.
http://goo.gl/DFMK8

 

 

Pictures that are literally worth over a million words

Check out this cool little tool: Google N-Grams

It shows a graphical representation of the frequency words used in books over a rangeof years. It is based on on all the books google has scanned into their database to date.

This TED talk is what turned me on to the project.

The project, the tool and the lecture are all quite entertaining.
Here are some graphs I created playing with the tool. Graphic data, especially that which is based on sentiment represented by our societies authors,  gives us amazing clues into how perception and reality intersect.












































































	


	
	






	
	

		
Quantify my language
	


	

		
oDesk Search Results
oDesk Search Results


The cost to scale a web application has changed considerably and therefor the way most people decide to plan out thier development strategies has changed as well. Not every web site is an enterprise solution and getting many sites or versions of sites out to the public quickly has become the greatest focus on a fledging comapanys mind.  Because scaling is so cheap and servers are so easy to set up with clouds I often hear, ‘lets deploy first and see what people think and if it gets big we will just through some servers at the problem, and improve the code as we grow.’  For the most part I agree, in the case that a complex problem arises an RPC call to some seperate more efficient code base, server or algorithm can be used.


Dividing up a site into 3 parts is key, re-inventing the wheel tasks, already built better then we can build it widget/api tasks, lets solve a new problem tasks. For example: login, and register has been developed 100 times over as well as setting up projects and DB connection settings. Getting a users feedback or publishing your content to all the social networks has been done beautifully by many companies that now give it away for almost nothing and are great to use, unless your new great idea has a core innocation in such technolgies. Then there are the problems out there that may be partially solved, but there is still quite a bit of technique and brain power needed to tackle your unique version of the problem to complete your companies mission, like improved search of billions of peices of data or 3 degree+ of sepration colculations. The mission is use what you can that already exists if its not your CORE business to own such IP (often times people think they have 5 core’s….they don’t) and use what you can to get that core going as quickly as possible. This isnt true for all web startup’s — but it is for most.


oDesk Feedbacks
oDesk Feedbacks


I will save more on that topic for a later post. Today I simply wanted to take the problem of getting started up a level or two and look at the first part of development for non developers.  I mean I personally know what I like to develop in, and it’s always good to ask someone with more experience then you what he/she thinks, but I all to often I find people that have inadvertenntly chosen the more expensive, more timley path and in the end they have a mess, a bill, and an almost incomplete site. They are then left with finding a batch of new developers who can jump right in and take over. I see this ALLOT and it breaks my heart. Is it jus a product of bad consulting? Is there a way to help get non developers a leg up and allow them to make good desicions quinatitatilbyl without a consultant telling them biasedly which way they can go to suit there own needs.


I know that oDesk, and other internet based freelance servcie is where allot of folks turn to so I went out and got some basic datapoint to start this conversation. Below are some charts I put togther using search scripts to get results sets from oDesk (whom I think should put out more repots on this matter to help out the public).  I simply tried to query things people may be wondering and get results sets for each lanuage respectivley. I was suprised to see how flat the results were.


 oDesk Search Results:







# high rated   feedback (2+ FB & 4+   stars)
total   feedbacks (4+ star)
most hours (2+ FB & 4+ stars)
$/hr of most hours (2+ FB & 4+ stars)
avg top 10 hours
avg $/hr of avg top 10 most hours
avg cheapst top 5
avg most exensive top 5
highest reqested rate




Python


3




6




606




$33.00




146




$25.34




$24.67




$28.10




$83.00






Ruby


12




9




2300




$16.00




663.5




$27.50




$14.93




$95.51




$230.00






PHP


64




37




4045




$30.00




1923.7




$26.44




$9.33




$75.62




$166.00






Java


12




15




712




$27.78




530.3




$26.13




$8.45




$54.09




$166.00






.net


10




14




2490




$19.44




641.6




$26.85




$11.55




$40.80




$111.00








Below is a list of some languages and how many hits came up when the languages marked with an X where combined in the search. I guess what I was going for here is how many people know what combinations of languages. For example people who posted a knowlagdge of all languages was 7, and those that claim to provide services to both python and ruby were 35.








Java




Python




Ruby




.net




PHP


Search       Hits






x








1427









x







246










x






300











x





1757












x




2397








Expand to show all results +



oDesk Results
oDesk Results


I will admit I dont like .net and I am not a fan of PHP as much as I used to be and although I enjoyed building FamilyOven.com in Java iterating over quick changes on the fly got really annoying having to compile and recompile my code. The pside of course is its great stability and easy to debug format. It’s just that scripted code suits my needs more these days. I put things together to push out new sites and stabiiy in the first year isnt that hard to do with server costs the way they are these days.  Python and ruby have allot of abstraction but for the first year I probably wont care, second year I will just pull hard tasks out like I explained in the first part of this email.


The data I gathered made me seriously wonder for a moment, am I just falling into the hype or is python and ruby as smooth, efficient and easy to develop sites quickly as great as it seems? I didnt jump on the ruby, python band wagon quickly. I created a few sites on it before I formed an opnion for sure.  I guess whats more important to notice here is how the hell is a non-technology/development person able is to decide on a languge to go with after talking to vendors and seeing results like these. I mean these data points love .net and php as giving off the impression that they are affordable, and popular.  I feel however that those that I know that use Ruby and Python are more innovative, fast, and love what they do more so then the others, not to mention cheaper in that they get more done more quickly. (or so it seems that way as of late) I would love to get some advice on ways to mine for data that respond with data that can statistically prove my sentiment isn’t just an opinion, a sort of freakenomics style of analysis if possible. I dont want to try to manipulate data to prove my point I just want to see if there are any true data points out there that confirm what my gut tells me.


Of course being that ruby and python are new compared to the other languages one could expect such unimpresive numbers and stats, I also think that being that they are so new to the common developer tool set it is impresive just how many people code on it already and how many plugins are available. One metric that could be usefull is expense to get started materials wise, free easy to use documentation, number of plugins, and time to start a basic project/site.


Python for instance is part of the Google standard languages, that speaks highly to me. It is supported by the Google App Engine and it has jython a frameowkr taht allows python to run in java runtime. That is mportant as well as java run time is more efficient then most things out there so you have flexability. Ruby has tons of plugins as well as python. and all scripting languages take almost no time to set up.  Also it is important to pointout that other then biz spark from MS getting a project developed is MS systems are not free to get started on from servers, to dev environments to documentation and support many MS product cost quite a bit which can get in the way of trial and error, innovation, speed to market, and keeping initial expenses down.


Please comment if you think I should account for a factor I missed or if youhave some data on your own.  If you have some crutial data points please list them and the source you were able to get it from. THe more straight data we get the more we can help people make better descisions rather then just using he told me so approach.


Here is some data about the job market in 2008. More data on jobs and how they relate to development would be helpful as well. i.e.  http://dotnetslackers.com/Community/blogs/xun/archive/2008/08/18/who-has-respect-in-web-development.aspx