Research into the evaluation of a data analysis contribution

in iamutopian •  last month

Repository

https://github.com/utopian-io/utopian.io

Introduction

As part of my role as a @utopian-io Community Manager, I have been set a task to research into how a company/organisation would evaluate a piece of data analysis. Ideally, this would involve finding out how these organisations review analysis work so that 'we' can collate the best practices used.

The goal of this is to provide input into the sets of guidelines for a @utopian-io contributor, moderator, and the community as a whole. Additionally, this research would also then feed into the moderation questionnaire which is used to evaluate and score an analysis contribution to @utopian-io.


An online approach

This is where I started my research into the subject, and due to not owning any relevant publications on the matter (not too surprising I hope!), it is also where the research for this blog ended. Using a variety of search terms on Google, such as:

'how to evaluate data analysis'

'evaluating data analysis'

'how to write a good data analysis'

'how to analyse an analysis' 😃

It quickly became clear that finding out how companies and organisations evaluate this type of work was not going to be easy. By far and away the most popular links returned from my searches were articles produced via educational institutions such as colleges and universities.

Thinking about it, it stands to reason that a business is unlikely to share the ins and outs of how they quantify and judge their internal analysis work, as this may well include tasks specific to an organisation. This in turn, may give clues on how they do business, and provide details of information that they would rather not share to competitors.

An educational establishment however, will be happy to provide insight into best practices, for its staff and students, in order to give them the best chance of producing a solid analysis. The drawback here is that these 'guides' are quite general and discuss more about best practices on how to produce the work, with less focus on how to evaluate the work done.

Despite not finding anything on how a large corporations score analytical endeavors, I think there is still much value to be gained from detailing what is sought after with regards to a good piece of analysis work. In presenting the general approach to producing a solid analysis, this should hopefully give insight into how it will then be evaluated. And I suppose the lead question a reviewer of the work will ask is, is this useful to the project it concerns, or not?

What is data analysis?

First though, a couple of definitions of data analysis:

Data analysis is a process of inspecting, cleansing, transforming, and modeling data with the goal of discovering useful information, informing conclusions, and supporting decision-making source

Data Analysis is the process of systematically applying statistical and/or logical techniques to describe and illustrate, condense and recap, and evaluate data. source

The process of evaluating data using analytical and logical reasoning to examine each component of the data provided. This form of analysis is just one of the many steps that must be completed when conducting a research experiment. Data from various sources is gathered, reviewed, and then analyzed to form some sort of finding or conclusion. There are a variety of specific data analysis method, some of which include data mining, text analytics, business intelligence, and data visualizations. source

Taking these three definitions of 'data analysis', there are a bunch of synonymous words that pop up frequently. 'Inspecting/examine', 'cleansing/condense', transform, discovering/finding, and an overlapping phrase including the word 'conclusion'.

From this, we can deduce that a successful or 'useful' analysis will involve carrying out at least some of these techniques. Depending on the type of analysis, covering all of this criteria may not be possible, but as most data analyses tend to follow the same course, it makes sense that the majority could be involved - the more the merrier, one might suggest?

Considerations/potential issues in data analysis

In order to help form further ideas of what may constitute a successful analysis, it is worth considering some of the potential pitfalls which can lower the quaility of the work undertaken. The following has been adapted from https://ori.hhs.gov

  • Drawing biased inferences: Leading into a piece of analysis work with an unbiased to potential outcome approach, should reduce the sway towards poorly formed conclusions.

  • Inappropriate subgroup analysis: Breaking down the analysis into smaller and smaller subgroups, in an attempt to find something deemed worthy of reporting, is likely not to yield anything of great interest, and sway further from the initial goals of the work.

  • Lack of clearly defined and objective outcome measurements: Poorly defined outcome objectives will set the tone for the work, even if the methods used in the analysis are of good quality.

  • Manner of presenting data: Full clarity on what is being presented, including clear headings, legends, and reasoning for displaying this particular piece of information will aid the work greatly.

  • Reliability and Validity: This relates to using the right 'stable' data sources and providing details to allow the analysis to be reproduced.

Summary

Whilst I did not find exactly what I set out to, there is a wealth of information available on the World Wide Web with regards to producing a successful analysis contribution. This information, as it has to some extent in the past, should be of some use in helping build a solid set of guidelines, and enable the expansion of the review questionnaire.

I can envision writing a couple more blog contributions with the #iamutopian tag once the questionniare/guidelines have been re-assessed, on this specific topic, and perhaps something like 'how to produce a successful analysis contribution to @utopian-io'.

Thanks for your time,

Asher [Analysis - Community Manager]

Resources

https://ori.hhs.gov/education/products/n_illinois_u/datamanagement/datopic.html

https://en.wikipedia.org/wiki/Data_analysis

http://www.businessdictionary.com/definition/data-analysis.html

https://review.utopian.io/

https://github.com/utopian-io/moderation-guidelines/blob/master/categories/analysis.md

Authors get paid when people like you upvote their post.
If you enjoyed what you read here, create your account today and start earning FREE STEEM!
Sort Order:  

Ironically, considering the post's topic, the current Blog category questionnaire is entirely irrelevant to our #iamutopian posts, so I'm going to just review this one as a text.

This definitely seems like laying in the groundwork for further blogging on the topic of how to judge and evaluate data analysis. You go through the basics, look at definitions, and create the building blocks for further study. In that, I found the post quite valuable and enlightening, as I know nothing about data analysis.

I did have a couple of issues when it comes to grammar, style, and proofreading. So let's head to the examples.

  • "as this may well include specific to task/organisation, give clues to how they do business, and provide details of information that they would rather not share to competitors, which could well include for reasons of privacy" is only part of a super long sentence that would have benefited from being broken up. "this may well include specific to task/organisation" is missing a word, I think? Maybe "details" after "include"? "provide details of information" seems to have a couple of redundant words. One of those is the "details" that missing earlier. "share to" should be "share with." Finally, "which could well include for reasons of privacy." What? Include what? Lemme try to edit this:
    "as this may well include details specific to task/organisation, give clues to how they do business, and provide information that they would rather not share with competitors. Another issue may well be privacy concerns."

  • "Despite not finding anything on how a large corporation score analytical endeavors" is grammatically incorrect. It should either be "how large corporations score" or "how a large corporation scores."

  • "gained on detailing" should be "gained from detailing."

More attention to details such as these would elevate this post from good to awesome. I hope to see that happen in the next posts in the series.

Your contribution has been evaluated according to Utopian policies and guidelines, as well as a predefined set of questions pertaining to the category.

To view those questions and the relevant answers related to your post, click here.


Need help? Write a ticket on https://support.utopian.io/.
Chat with us on Discord.
[utopian-moderator]

·

Hi @didic

Thanks for the detailed review of my first #iamutopian blog submission.

English was never my strongest discipline (I'm English :) ), I guess it's improved gradually over the years, but still work to do.

I have now edited the post with regard to the suggestions and errors detailed by you.

Cheers!

Asher

·

Thank you for your review, @didic!

So far this week you've reviewed 23 contributions. Keep up the good work!

Whilst I did not find exactly what I set out to, there is a wealth of information available on the World Wide Web with regards to producing a successful analysis contribution.
@abh12345

Would a look at Palantir help, Asher?

·

interesting! Do you use their tools or have experience with the company? The public information is unfortunately mostly shiny PR without many technical details.

·
·

The public information is unfortunately mostly shiny PR without many technical details.
@crokkon

Palantir attempts to stay as much under the radar, as possible. 😊

Do you use their tools or have experience with the company?
@crokkon

An acquaintance worked there.

·
·
·

Ok, that clearly goes deeper, thanks for the link! :)

·

I will get a coffee and have a listen....

My understanding states, data analysis is about showing the correct interpretation of an organisation. The analysis starts from the scratch till the top. The usage ,wastage, surplus everything need to be considered to show the raeal net worth of the company.

Oopss..you already doing it smartly in engagement league. So i think they reach out to the best to score on Data Analysis.

Posted using Partiko Android

·

Ahh thanks man, much appreciated!

I do little 'conclusion' type analysis for the league, but there is a fair amount of gathering going on!

I just created content for a 20 week pathway, introduction to data analytics. While researching the content I was wondering how I was going to fit it all into 20 weeks. there is just so much, so many tool and technologies available now. One topic I did not cover is how to analyse the analysis 😉

·

how to analyse the analysis

One of my google searches :D

Do you think it's right to say that it will be analysed based on meeting the definition of analysis work to some extent? And uncovering something useful?

·
·

uncovering something useful

data analysis is most useful when you have direct questions you want answered, although exploratory analysis could lead to those questions

·
·
·

When you know what is expected and whzt to give.

Posted using Partiko Android

·
·
·

Shhhh, don't tell that to my ecologist colleagues. ;)

Data analysis have great force as you explained

·

They can be very useful, thanks!

Wonderful nice post to share thanks you did not vote me any way one vote by me but I like unique articles thanks to support me as well

·

Thank you very much for the support :D

·
·

My pleasure you most welcome

Data analysis is a great force in the modern economy and your research was helful in knowing more about it

Hi @abh12345!

Your post was upvoted by @steem-ua, new Steem dApp, using UserAuthority for algorithmic post curation!
Your UA account score is currently 6.458 which ranks you at #160 across all Steem accounts.
Your rank has improved 2 places in the last three days (old rank 162).

In our last Algorithmic Curation Round, consisting of 266 contributions, your post is ranked at #4.

Evaluation of your UA score:
  • You've built up a nice network.
  • The readers appreciate your great work!
  • Good user engagement!

Feel free to join our @steem-ua Discord server