If you’ve ever tried to execute a complex analysis, you know the feeling of getting stuck. It’s a normal part of any analysis to need time to reflect, but what if you find yourself struggling longer than usual? In these cases, you may feel stumped.
I’ve spoken with several analysts about this feeling, and there seems to be a consensus on the reasons why we get stuck – there are 3 of them. The challenge is that even when we’re aware of the reasons, we become blind to them in the intensity of the analysis. The way to get unstuck is to be actively conscious of the reason.
The most common reason analysts get “stuck” in analysis is that they don’t understand the relationships between the dimensions in the specific underlying dataset. This means they can usually clear any confusion by revisiting each dimension, one-by-one, and playing with filters to test their comprehension, as well as talking to colleagues who have more experience with the data.
3 Reasons We Get Stuck
Not understanding the dimensions of the dataset may be the leading cause for getting stuck, but it’s not the only one. Together, the 3 leading reasons are:
- Not understanding the relationships of the dimensions in the specific dataset.
- Lack of experience with the analytic tool (Excel, Tableau).
- Too little knowledge of prerequisite analyses.
It’s important to keep these reasons in mind so you can act on them when the time comes. When we’re faced with a new challenge, we jump right into the analysis in an effort to quickly show that we know what we’re doing — to understand the stakes, show our capabilities, and urgently find a solution.
The pressure from coworkers or superiors to perform may make us blind to the reason why we’re stuck, and this can lead to a loss of confidence and worse performance over time.
But you have to know what the issue is before you can solve it. Let’s look at these 3 reasons in closer detail.
1. Not Understanding the Relationships of the Dimensions in the Specific Dataset
As most analysts know, every dataset is different. The threat to an analyst is that she/he tries to apply assumptions about relationships from previous datasets to the current one.
Some datasets are highly transactional, which means each attribute is distinctly different from the others, and the only similarity is the primary key.
However, some datasets are more hierarchical, wherein one dimension may be entirely dependent on the results of another dimension. An example of this occurs in surveys, where a respondent may be led to different questions based on earlier responses. This most often shows up with phrases such as “If yes, go to question X. If no, go to question Y.”
As you can imagine, these relationships become complex. An analyst who does not deeply understand the dataset with which he or she is working may become stuck when the results of an analysis don’t turn out as planned, and this can easily result from a lack of dimension understanding.
2. Lack of Experience with Analytic Tools
For beginner-level analysts looking to improve their analytic skills, a lack of experience with different tools is the most common cause for getting stuck.
It’s common to feel confident about your intelligence and ability to succeed in data as a concept, such as in school, but it’s rare that analysts with less exposure to various tools have had enough experience to know how to manage large analyses with only basic skills in Microsoft Excel.
Even if you’ve had some experience with another tool, it’s unlikely you have a wholistic enough grasp of the strengths and weaknesses of each tool to exploit them when faced with various challenges in an advanced analysis.
This is how a lack of experience with analytic tools can spell disaster for an analytic workflow. Young analysts find themselves frustrated, but we’ll look at how to fix this in the next section.
3. Too Little Knowledge with Prerequisite Analyses
Analysis is a big term that encompasses many different types. When we spend a lot of time with one type, we are less familiar with others. Then we attempt them and get stuck because we haven’t built a good base of knowledge and exposure to related analyses.
Take a look at this picture that shows the different types of analyses:
As you can see, there are several different types of analysis. Imagine you are very familiar with time series data, for example, and have a lot of experience with forecasting.
If someone then asks you to perform clustering methodologies, you would likely have a hard time with an advanced project.
This is because you lack the fundamental understanding of prerequisite, simpler models. In other words, if you’re feeling stuck, it may be because you’re trying to solve a problem for which you don’t have the right experience.
While that may sound simple, it’s a truth that’s very easy to forget when you’re caught up in the complexity of an analytical challenge.
How To Get Unstuck: Be Aware of the Cause
The challenge with feeling stuck is that we often become blind to the reason. In the heat of analysis, just like in an argument, we can loose our way. The best way to solve our problem is to become aware of the challenge we’re facing. But that’s easier said than done. Here are a few ways you can clear your head.
1. Review each dimension one-by-one
It may feel painful, but by reviewing your understanding of each of the dimensions at play, you can reinitiate your understanding of the dataset as a whole. In my experience, when this is the source of my confusion, I figure out what was stumping me long before reviewing each field.
2. Try reverting to a data tool you know well
Most analysts are working with tools and coding languages that far outperform Excel. But at the end of the day, Excel shows us the data in the most fundamental way: in tables made of rows and columns.
Often when I’m working with something complex in a visualization software such as Tableau or PowerBI, I take a small sample of the dataset and plug it into Excel to get a renewed view of the challenge.
3. Perform very simple versions of your analysis
When your confusion comes from a complex form of a specific type of analysis, try going back to basics and performing the simplest version of that analysis possible, but using the data from the relevant dataset.
This can be difficult to do. Taking a full step back from the task at hand requires shifting gears, and change. Most people struggle to do this, myself included. But when you do, your brain gets a break.
For example, if you’re running a complex k-means cluster analysis with 10 different cluster centers and find yourself stuck, you could try going down to 3 clusters. Or you might try plotting all of the data and simply hand-picking data centers (although this is not a long term solution for data clusters).
I think you’ll see that it allows you to better conceptualize the issue you’re having with the large dataset. By understanding the simple form of the analysis, the complex form will become clearer.
4. Get some fresh air
Sometimes when we’re too focused on a topic, we’re not only mentally stuck, but also physically. It’s a vicious cycle that traps your body and mind. A simple way to get some perspective is to get outside and get some fresh air!
5. Talk to analysts who know the dataset well
The good news about data analysis is that the community is generally supportive. If you’re stuck and you cannot identify the reason, you may just need to discuss with colleagues who know the dataset better. Sometimes there are things that you don’t know you don’t know, and only talking to people with experience can get you over that hurdle.
