Without actual data it is hard to answer the question but I guess you are looking for something like this:
Top15['Citable docs per Capita'].corr[Top15['Energy Supply per Capita']]
That calculates the correlation between your two columns 'Citable docs per Capita'
and 'Energy Supply per Capita'
.
To give an example:
import pandas as pd
df = pd.DataFrame[{'A': range[4], 'B': [2*i for i in range[4]]}]
A B
0 0 0
1 1 2
2 2 4
3 3 6
Then
df['A'].corr[df['B']]
gives 1
as expected.
Now, if you change a value, e.g.
df.loc[2, 'B'] = 4.5
A B
0 0 0.0
1 1 2.0
2 2 4.5
3 3 6.0
the command
df['A'].corr[df['B']]
returns
0.99586
which is still close to 1, as expected.
If you apply .corr
directly to your dataframe, it will return all pairwise correlations between your columns; that's why you then observe 1s
at the diagonal of your matrix [each column is perfectly correlated with itself].
df.corr[]
will therefore return
A B
A 1.000000 0.995862
B 0.995862 1.000000
In the graphic you show, only the upper left corner of the correlation matrix is represented [I assume].
There can be cases, where you get NaN
s in your solution - check this post for an example.
If you want to filter entries above/below a certain threshold, you can check this question. If you want to plot a heatmap of the correlation coefficients, you can check this answer and if you then run into the issue with overlapping axis-labels check the following post.
View Discussion
Improve Article
Save Article
View Discussion
Improve Article
Save Article
In this article, we will discuss how to calculate the correlation between two columns in pandas
Correlation is used to summarize the strength and direction of the linear association between two quantitative variables. It is denoted by r and values between -1 and +1. A positive value for r indicates a positive association, and a negative value for r indicates a negative association.
By using corr[] function we can get the correlation between two columns in the dataframe.
Syntax:
dataframe[‘first_column’].corr[dataframe[‘second_column’]]
where,
- dataframe is the input dataframe
- first_column is correlated with second_column of the dataframe
Example 1: Python program to get the correlation among two columns
Python3
import
pandas as pd
data
=
pd.DataFrame[{
"column1"
: [
12
,
23
,
45
,
67
],
"column2"
: [
67
,
54
,
32
,
1
],
"column3"
: [
34
,
23
,
56
,
23
]
}
]
print
[data]
print
[data[
'column1'
].corr[data[
'column2'
]]]
print
[data[
'column2'
].corr[data[
'column3'
]]]
print
[data[
'column1'
].corr[data[
'column3'
]]]
Output:
column1 column2 column3 0 12 67 34 1 23 54 23 2 45 32 56 3 67 1 23 -0.9970476685163736 0.07346999975265099 0.0
It is also possible to get element-wise correlation for numeric valued columns using just corr[] function.
Syntax:
dataset.corr[]
Example 2: Get the element-wise correlation
Python3
import
pandas as pd
data
=
pd.DataFrame[{
"column1"
: [
12
,
23
,
45
,
67
],
"column2"
: [
67
,
54
,
32
,
1
],
"column3"
: [
34
,
23
,
56
,
23
]
}
]
print
[data.corr[]]
Output:
column1 column2 column3 column1 1.000000 -0.997048 0.00000 column2 -0.997048 1.000000 0.07347 column3 0.000000 0.073470 1.00000