fredag 16 juni 2017

Think Like a Statistician – Without the Math

FlowingData:
I call myself a statistician, because, well, I’m a statistics graduate student. However, ask me specific questions about hypothesis tests or required sampling size, and my answer probably won’t be very good.

The other day I was trying to think of the last time I did an actual hypothesis test or formal analysis. I couldn’t remember. I actually had to dig up old course listings to figure out when it was. It was four years ago during my first year of graduate school. I did well in those courses, and I’m confident I could do that stuff with a quick refresher, but it’s a no go off the cuff. It’s just not something I do regularly.

Instead, the most important things I’ve learned are less formal, but have proven extremely useful when working/playing with data. Here they are in no particular order.

Read more...

söndag 11 juni 2017

Experterna hade fel – Macron är ostoppbar

Foto: Alexander Zemlianichenko/AP
Teresa Küchler | SvD:
En stor väljarsondering inför söndagens första omgång av valet till nationalförsamlingen i Paris, alltså det franska parlamentets underhus, förutspår att president Macrons liberala rörelse ”Framåt för republiken!” får omkring 400 av de totalt 577 stolarna i kammaren. Det blir i så fall den största majoriteten sedan slutet av 60-talet.

UK general election: Five steps to make sense of the latest polls

Significance magazine: 
Unlike the 2015 general election, when the polls were essentially static (and wrong) throughout, the 2017 general election has seen some of the most extraordinary volatility in the polls that I can remember. If you are a Conservative supporter, the narrowing lead over Labour must be leading to anxiety and changed underwear. If you are a Labour supporter, you are probably starting to dream “can we? will we?!”

Read more....

lördag 10 juni 2017

The U.K. Election Wasn’t That Much Of A Shock

FiveThirtyEight:
Theresa May’s loss was dramatic, but polls had shown her majority at some risk.

Despite betting markets and expert forecasts that predicted Theresa May’s Conservatives to win a large majority in the U.K. parliamentary elections, the Tories instead lost ground on Thursday, resulting in a hung parliament. As we write this in the early hours of Friday morning, Conservatives will end up with either 318 or 319 seats, down from the 330 that the Tories had in the previous government. A majority officially requires 326 seats.

fredag 9 juni 2017

Skilltest: Linear Regression

Analytics Vidhya:
Linear Regression is possibly the most widely used technique in Machine Learning. It is also the most researched technique in academia. You can't call yourself a data science aspirant until you know Linear Regression well. This skill test is specially designed for you to test your knowledge on linear regression techniques.

torsdag 8 juni 2017

The Three Scenarios For The U.K. Election

FiveThirtyEight:
On the morning of the U.S. presidential election, we pointed out that there were three scenarios for what might transpire that night, each of which were about equally likely. In Scenario No. 1, the polls would be spot-on; Hillary Clinton would win narrowly, with a 3-to-4 percentage point popular victory and somewhere on the order of 300 electoral votes. In Scenario No. 2, Clinton would outperform her polls, leading to a near-landslide victory and possible wins in states such as Arizona and Georgia which had traditionally favored Republicans. And in Scenario No. 3, Donald Trump would beat his polls; because the Electoral College favored Trump, even a small polling error in his favor would probably be enough to make him president. Scenario No. 3 is the one that transpired, but it wasn’t any more or less likely than the other two.

Read more....

söndag 4 juni 2017

Are The U.K. Polls Skewed?

FiveThirtyEight:
Bettors expect the polls to underrate Conservatives again. If they underrate Labour instead, Theresa May’s majority is at risk.

In April, when U.K. Prime Minister Theresa May called for a “snap” general election for June 8, polls showed her Conservatives with an average lead of 17 percentage points over Labour. Such a margin would translate to a giant majority for Conservatives: perhaps as many as 400 of the 650 seats in Parliament. (Conservatives currently control 330 seats; 326 are needed for a majority.) After several unpredictable years in U.K. politics — marked by Conservatives unexpectedly winning a majority in the 2015 general election, the successful Brexit referendum, and David Cameron’s decision to resign as prime minister and Conservative leader — such a result promised to provide May with a mandate as she negotiated the terms of the U.K.’s exit from the EU.

Read more....

fredag 2 juni 2017

SCB ger ingen tröst

Nyheter (Ekot) | Sveriges Radio:
Två partier rusar och ett parti rasar i opinionen. Men trots dramatiska förändringar fortsätter väljarna att trotsa partiledningarnas ideer om hur Sverige ska styras. Ingen partiledning som tänker längre än morgondagen kan känna riktig glädje över SCB:s senaste staplar, enligt Ekots politiske kommentator Tomas Ramberg.

The seven deadly sins of statistical misinterpretation, and how to avoid them

The Conversation:
Statistics is a useful tool for understanding the patterns in the world around us. But our intuition often lets us down when it comes to interpreting those patterns. In this series we look at some of the common mistakes we make and how to avoid them when thinking about statistics, probability and risk.

1. Assuming small differences are meaningful

Many of the daily fluctuations in the stock market represent chance rather than anything meaningful. Differences in polls when one party is ahead by a point or two are often just statistical noise.

You can avoid drawing faulty conclusions about the causes of such fluctuations by demanding to see the “margin of error” relating to the numbers.

If the difference is smaller than the margin of error, there is likely no meaningful difference, and the variation is probably just down to random fluctuations.

onsdag 31 maj 2017

Predicting the UK’s snap general election

Significance magazine:
I have decided to build a model to try to predict the results of the upcoming snap general election in the UK. I'm sure there will be many people attempting this, from various perspectives and using different modelling approaches. But I have set out to develop a fairly simple (though, hopefully, reasonable) model. In the process of describing this to you, I hope to shed some light on how statisticians build predictive models.

tisdag 30 maj 2017

Så förklarar Gapminder svåra saker med enkla bilder

Gapminders tjänst Dollar Street visar att hem i olika delar världen
 ser ungefär likadana ut för familjer på samma inkomstnivå.
Computer Sweden:
Om du frågar de flesta i Sverige om namnet på en person som är duktig på att visualisera data så lär svaret bli den nyligen bortgångne Hans Rosling. Men han jobbade inte på egen hand, utan tillsammans med sonen Ola Rosling och svärdottern Anna Rosling Rönnlund under 18 år. Nu fortsätter Ola och Anna på den inslagna vägen med Stiftelsen Gapminder som de grundade 2005, tillsammans med Hans Rosling.

Under ett föredrag nyligen på konferensen Data Storytellers i Stockholm demonstrerade Anna Rosling Rönnlund tjänsten Dollar Street som Gapminder byggt. Den går ut på att beskriva levnadsförhållanden för människor över hela världen, baserat på inkomst. Under föredraget visar hon att det ser ungefär likadant ut i hem över hela världen, för familjer på samma inkomstnivåer.

Läs mer....

lördag 27 maj 2017

Truth and statistics: How to find out what people really think

The Economist:
TO MANY people Big Data is less shiny than it was a year ago. After Hillary Clinton’s defeat at the hands of Donald Trump, her vaunted analytics team took much of the blame for failing to spot warnings in the midwestern states that cost her the presidency. But according to research by Seth Stephens-Davidowitz, a former data scientist at Google, Mrs Clinton’s real mistake was not to rely too much on newfangled statistics, but rather too little.

onsdag 24 maj 2017

8 out of 10 cats fear statistics – AI doesn't have this problem

Use and abuse of figures
The Register:
If statistics were a human being, it would have been in deep therapy all of its 350-year life. The sessions might go like this:

Statistics: "Everyone hates me."

Pause.

Therapist: "I'm sure it's not everyone..."

Statistics: "And they misunderstand me."

Pause.

Therapist: "Sorry, I didn't quite get what you meant there..."

The problem is that statistics are misunderstood by the majority of the population and most people hate what they don't understand. Think of the well-known expressions: "Lies, damn lies and statistics" and "The government uses statistics as a drunkard uses a lamppost; more for support than for illumination."

måndag 22 maj 2017

De tar Hans Roslings hyllade kommunikationskoncept vidare

Från Resumé:
Han fick tung statistik att flyga genom staplade toapappersrullar och kaffekoppar. Bakom avlidne professor Hans Roslings innovativa sätt att presentera globala förändringar står sonen Ola Rosling och sonhustrun Anna Rosling Rönnlund.

På Ted X i Singapore kunde åhörarna vila ögonen på toalettpappersrullar och samtidigt lära sig fakta om hur världens befolkningstillväxt utvecklar sig. Men den gången i november 2015 var inte den första som Hans Rosling skapade uppmärksamhet för sina scenframträdanden.

Vad förklarar SD:s framgång?

Ekonomistas
Genomsnittligt väljarstöd i nationella parlament för högerpopulistiska partier i EU samt Island,
Norge, Schweiz, Serbien och Montenegro. Diagram från Timbros index över auktoritär populism 2016.

Få saker i vår samtid diskuteras lika flitigt som hur vi bäst kan förstå de nationalistiska strömningar som sköljer över västvärlden. Trumps valseger och Brexit överrumplade många, men faktum är att högerpopulistiska partier i Europa varit på frammarsch alltsedan 1980-talet (se figur nedan). Här i Sverige har SD ungefärligen fördubblat sitt väljarstöd i varje riksdagsval sedan 1988. Efter Macrons seger i Frankrike tycks många nu hoppas att kulmen har nåtts. Men utan att förstå drivkrafterna bakom högerpopulismens framväxt är det naturligtvis omöjligt att sia om den framtida utvecklingen. Jag vågar inte ge mig i kast med att förklara de bakomliggande drivkrafterna i alla länder, men när det gäller den svenska utveckligen undrar jag om inte en mycket enkel förklaringsmodell med tre beståndsdelar räcker ganska långt.

Läs mer....

torsdag 18 maj 2017

Statistics NZ CEO Liz MacPherson: Into tomorrow with information from today

Photo by Divina Paredes
CIO from IDG:
‘We are absolutely focused on unleashing the power of data to change lives.’

“You are already a statistic, you are the first woman Government Statistician.”

Someone said this to Liz MacPherson when she took on the top role at Statistics New Zealand three years ago.

And, indeed, the wall in her new office was filled with photos of her predecessors, all males.

Read more....

tisdag 16 maj 2017

Real or Fake News? Let Statistics Help: 7 Questions to Ask

This is Statistics:
There is a lot of discussion today about whether the stories we see in the news are real or fake. Statistical thinking can help you assess the validity of reports, claims from a new study, or other conclusions flashing through your social media feed. Here are a few tips from statisticians – experts in the scientific discipline of learning from data – for how to separate fact from fiction, science from salesmanship, precision from propaganda.

lördag 13 maj 2017

Sparse graphs using exchangeable random measures



Speakers: Francois Caron (University of Oxford, UK) and Emily B Fox (University of Washington, Seattle, USA)

Statistical network modelling has focused on representing the graph as a discrete structure, namely the adjacency matrix. When assuming exchangeability of this array—which can aid in modelling, computations, and theoretical analysis—the Aldous-Hoover theorem informs us that the graph is necessarily either dense or empty. We instead consider representing the graph as an exchangeable random measure and appeal to the Kallenberg representation theorem for this object. We explore using completely random measures (CRMs) to define the exchangeable random measure and we show how our CRM construction enables us to achieve sparse graphs while maintaining the attractive properties of exchangeability. We relate the sparsity of the graph to the Lévy measure defining the CRM. For a specific choice of CRM, our graphs can be tuned from dense to sparse on the basis of a single parameter. We present a scalable Hamiltonian Monte Carlo algorithm for posterior inference, which we use to analyse network properties in a range of real data sets, including networks with hundreds of thousands of nodes and millions of edges.