Introduction¶

In [1]:

import pandas as pd

In [2]:

df = pd.read_csv("survey_results_public.csv")

In [3]:

df

Out[3]:

	Respondent	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	...	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
0	1	I am a student who is learning to code	Yes	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	...	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
1	2	I am a student who is learning to code	No	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	...	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	3	I am not primarily a developer, but I write co...	Yes	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	...	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
3	4	I am a developer by profession	No	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	...	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
4	5	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	...	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88878	88377	NaN	Yes	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	Not employed, and not looking for work	Canada	No	Primary/elementary school	NaN	...	NaN	Tech articles written by other developers;Tech...	NaN	Man	No	NaN	NaN	No	Appropriate in length	Easy
88879	88601	NaN	No	Never	The quality of OSS and closed source software ...	NaN	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88880	88802	NaN	No	Never	NaN	Employed full-time	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88881	88816	NaN	No	Never	OSS is, on average, of HIGHER quality than pro...	Independent contractor, freelancer, or self-em...	NaN	NaN	NaN	NaN	...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88882	88863	NaN	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, and not looking for work	Spain	Yes, full-time	Professional degree (JD, MD, etc.)	Computer science, computer engineering, or sof...	...	Somewhat less welcome now than last year	Tech articles written by other developers;Indu...	18.0	Man	No	Straight / Heterosexual	Hispanic or Latino/Latina;White or of European...	No	Appropriate in length	Easy

88883 rows × 85 columns

In [4]:

df.shape #it's an attribute and not a method

Out[4]:

(88883, 85)

In [5]:

df.info() #now this is a method

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 88883 entries, 0 to 88882
Data columns (total 85 columns):
 #   Column                  Non-Null Count  Dtype  
---  ------                  --------------  -----  
 0   Respondent              88883 non-null  int64  
 1   MainBranch              88331 non-null  object 
 2   Hobbyist                88883 non-null  object 
 3   OpenSourcer             88883 non-null  object 
 4   OpenSource              86842 non-null  object 
 5   Employment              87181 non-null  object 
 6   Country                 88751 non-null  object 
 7   Student                 87014 non-null  object 
 8   EdLevel                 86390 non-null  object 
 9   UndergradMajor          75614 non-null  object 
 10  EduOther                84260 non-null  object 
 11  OrgSize                 71791 non-null  object 
 12  DevType                 81335 non-null  object 
 13  YearsCode               87938 non-null  object 
 14  Age1stCode              87634 non-null  object 
 15  YearsCodePro            74331 non-null  object 
 16  CareerSat               72847 non-null  object 
 17  JobSat                  70988 non-null  object 
 18  MgrIdiot                61159 non-null  object 
 19  MgrMoney                61157 non-null  object 
 20  MgrWant                 61232 non-null  object 
 21  JobSeek                 80555 non-null  object 
 22  LastHireDate            79854 non-null  object 
 23  LastInt                 67155 non-null  object 
 24  FizzBuzz                71344 non-null  object 
 25  JobFactors              79371 non-null  object 
 26  ResumeUpdate            77877 non-null  object 
 27  CurrencySymbol          71392 non-null  object 
 28  CurrencyDesc            71392 non-null  object 
 29  CompTotal               55945 non-null  float64
 30  CompFreq                63268 non-null  object 
 31  ConvertedComp           55823 non-null  float64
 32  WorkWeekHrs             64503 non-null  float64
 33  WorkPlan                68914 non-null  object 
 34  WorkChallenge           68141 non-null  object 
 35  WorkRemote              70284 non-null  object 
 36  WorkLoc                 70055 non-null  object 
 37  ImpSyn                  71779 non-null  object 
 38  CodeRev                 70390 non-null  object 
 39  CodeRevHrs              49790 non-null  float64
 40  UnitTests               62668 non-null  object 
 41  PurchaseHow             61108 non-null  object 
 42  PurchaseWhat            62029 non-null  object 
 43  LanguageWorkedWith      87569 non-null  object 
 44  LanguageDesireNextYear  84088 non-null  object 
 45  DatabaseWorkedWith      76026 non-null  object 
 46  DatabaseDesireNextYear  69147 non-null  object 
 47  PlatformWorkedWith      80714 non-null  object 
 48  PlatformDesireNextYear  77443 non-null  object 
 49  WebFrameWorkedWith      65022 non-null  object 
 50  WebFrameDesireNextYear  62944 non-null  object 
 51  MiscTechWorkedWith      59586 non-null  object 
 52  MiscTechDesireNextYear  64511 non-null  object 
 53  DevEnviron              87317 non-null  object 
 54  OpSys                   87851 non-null  object 
 55  Containers              85366 non-null  object 
 56  BlockchainOrg           48175 non-null  object 
 57  BlockchainIs            60165 non-null  object 
 58  BetterLife              86269 non-null  object 
 59  ITperson                87141 non-null  object 
 60  OffOn                   86663 non-null  object 
 61  SocialMedia             84437 non-null  object 
 62  Extraversion            87305 non-null  object 
 63  ScreenName              80486 non-null  object 
 64  SOVisit1st              83877 non-null  object 
 65  SOVisitFreq             88263 non-null  object 
 66  SOVisitTo               88086 non-null  object 
 67  SOFindAnswer            87816 non-null  object 
 68  SOTimeSaved             86344 non-null  object 
 69  SOHowMuchTime           68378 non-null  object 
 70  SOAccount               87828 non-null  object 
 71  SOPartFreq              74692 non-null  object 
 72  SOJobs                  88066 non-null  object 
 73  EntTeams                87841 non-null  object 
 74  SOComm                  88131 non-null  object 
 75  WelcomeChange           85855 non-null  object 
 76  SONewContent            69560 non-null  object 
 77  Age                     79210 non-null  float64
 78  Gender                  85406 non-null  object 
 79  Trans                   83607 non-null  object 
 80  Sexuality               76147 non-null  object 
 81  Ethnicity               76668 non-null  object 
 82  Dependents              83059 non-null  object 
 83  SurveyLength            86984 non-null  object 
 84  SurveyEase              87081 non-null  object 
dtypes: float64(5), int64(1), object(79)
memory usage: 57.6+ MB

In [6]:

#object generally means string

In [7]:

pd.set_option("display.max_columns",85) #changes the option in jupyter on how many columns are shown
pd.set_option("display.max_rows",85) #changes the option in jupyter on how many rows are shown

In [8]:

schema_df = pd.read_csv("survey_results_schema.csv")

In [9]:

schema_df

Out[9]:

	Column	QuestionText
0	Respondent	Randomized respondent ID number (not in order ...
1	MainBranch	Which of the following options best describes ...
2	Hobbyist	Do you code as a hobby?
3	OpenSourcer	How often do you contribute to open source?
4	OpenSource	How do you feel about the quality of open sour...
5	Employment	Which of the following best describes your cur...
6	Country	In which country do you currently reside?
7	Student	Are you currently enrolled in a formal, degree...
8	EdLevel	Which of the following best describes the high...
9	UndergradMajor	What was your main or most important field of ...
10	EduOther	Which of the following types of non-degree edu...
11	OrgSize	Approximately how many people are employed by ...
12	DevType	Which of the following describe you? Please se...
13	YearsCode	Including any education, how many years have y...
14	Age1stCode	At what age did you write your first line of c...
15	YearsCodePro	How many years have you coded professionally (...
16	CareerSat	Overall, how satisfied are you with your caree...
17	JobSat	How satisfied are you with your current job? (...
18	MgrIdiot	How confident are you that your manager knows ...
19	MgrMoney	Do you believe that you need to be a manager t...
20	MgrWant	Do you want to become a manager yourself in th...
21	JobSeek	Which of the following best describes your cur...
22	LastHireDate	When was the last time that you took a job wit...
23	LastInt	In your most recent successful job interview (...
24	FizzBuzz	Have you ever been asked to solve FizzBuzz in ...
25	JobFactors	Imagine that you are deciding between two job ...
26	ResumeUpdate	Think back to the last time you updated your r...
27	CurrencySymbol	Which currency do you use day-to-day? If your ...
28	CurrencyDesc	Which currency do you use day-to-day? If your ...
29	CompTotal	What is your current total compensation (salar...
30	CompFreq	Is that compensation weekly, monthly, or yearly?
31	ConvertedComp	Salary converted to annual USD salaries using ...
32	WorkWeekHrs	On average, how many hours per week do you work?
33	WorkPlan	How structured or planned is your work?
34	WorkChallenge	Of these options, what are your greatest chall...
35	WorkRemote	How often do you work remotely?
36	WorkLoc	Where would you prefer to work?
37	ImpSyn	For the specific work you do, and the years of...
38	CodeRev	Do you review code as part of your work?
39	CodeRevHrs	On average, how many hours per week do you spe...
40	UnitTests	Does your company regularly employ unit tests ...
41	PurchaseHow	How does your company make decisions about pur...
42	PurchaseWhat	What level of influence do you, personally, ha...
43	LanguageWorkedWith	Which of the following programming, scripting,...
44	LanguageDesireNextYear	Which of the following programming, scripting,...
45	DatabaseWorkedWith	Which of the following database environments h...
46	DatabaseDesireNextYear	Which of the following database environments h...
47	PlatformWorkedWith	Which of the following platforms have you done...
48	PlatformDesireNextYear	Which of the following platforms have you done...
49	WebFrameWorkedWith	Which of the following web frameworks have you...
50	WebFrameDesireNextYear	Which of the following web frameworks have you...
51	MiscTechWorkedWith	Which of the following other frameworks, libra...
52	MiscTechDesireNextYear	Which of the following other frameworks, libra...
53	DevEnviron	Which development environment(s) do you use re...
54	OpSys	What is the primary operating system in which ...
55	Containers	How do you use containers (Docker, Open Contai...
56	BlockchainOrg	How is your organization thinking about or imp...
57	BlockchainIs	Blockchain / cryptocurrency technology is prim...
58	BetterLife	Do you think people born today will have a bet...
59	ITperson	Are you the "IT support person" for your family?
60	OffOn	Have you tried turning it off and on again?
61	SocialMedia	What social media site do you use the most?
62	Extraversion	Do you prefer online chat or IRL conversations?
63	ScreenName	What do you call it?
64	SOVisit1st	To the best of your memory, when did you first...
65	SOVisitFreq	How frequently would you say you visit Stack O...
66	SOVisitTo	I visit Stack Overflow to... (check all that a...
67	SOFindAnswer	On average, how many times a week do you find ...
68	SOTimeSaved	Think back to the last time you solved a codin...
69	SOHowMuchTime	About how much time did you save? If you're no...
70	SOAccount	Do you have a Stack Overflow account?
71	SOPartFreq	How frequently would you say you participate i...
72	SOJobs	Have you ever used or visited Stack Overflow J...
73	EntTeams	Have you ever used Stack Overflow for Enterpri...
74	SOComm	Do you consider yourself a member of the Stack...
75	WelcomeChange	Compared to last year, how welcome do you feel...
76	SONewContent	Would you like to see any of the following on ...
77	Age	What is your age (in years)? If you prefer not...
78	Gender	Which of the following do you currently identi...
79	Trans	Do you identify as transgender?
80	Sexuality	Which of the following do you currently identi...
81	Ethnicity	Which of the following do you identify as? Ple...
82	Dependents	Do you have any dependents (e.g., children, el...
83	SurveyLength	How do you feel about the length of the survey...
84	SurveyEase	How easy or difficult was this survey to compl...

In [10]:

df.head(12)

Out[10]:

	Respondent	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
0	1	I am a student who is learning to code	Yes	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	4	10	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript;Python	C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL	SQLite	MySQL	MacOS;Windows	Android;Arduino;Windows	Django;Flask	Flask;jQuery	Node.js	Node.js	IntelliJ;Notepad++;PyCharm	Windows	I do not use containers	NaN	NaN	Yes	Fortunately, someone else has that title	Yes	Twitter	Online	Username	2017	A few times per month or weekly	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	31-60 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
1	2	I am a student who is learning to code	No	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Developer, desktop or enterprise applications;...	NaN	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Financial performance or funding status of the...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C++;HTML/CSS;Python	C++;HTML/CSS;JavaScript;SQL	NaN	MySQL	Windows	Windows	Django	Django	NaN	NaN	Atom;PyCharm	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	Yes	Instagram	Online	Username	2017	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	3	I am not primarily a developer, but I write co...	Yes	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	Taught yourself a new language, framework, or ...	100 to 499 employees	Designer;Developer, back-end;Developer, front-...	3	22	1	Slightly satisfied	Slightly satisfied	Not at all confident	Not sure	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	THB	Thai baht	23000.0	Monthly	8820.0	40.0	There's no schedule or spec; I work on what se...	Distracting work environment;Inadequate access...	Less than once per month / Never	Home	Average	No	NaN	No, but I think we should	Not sure	I have little or no influence	HTML/CSS	Elixir;HTML/CSS	PostgreSQL	PostgreSQL	NaN	NaN	NaN	Other(s):	NaN	NaN	Vim;Visual Studio Code	Linux-based	I do not use containers	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
3	4	I am a developer by profession	No	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
4	5	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Academic researcher;Developer, desktop or ente...	16	14	9	Very dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	No	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	UAH	Ukrainian hryvnia	NaN	NaN	NaN	55.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	A little above average	Yes, because I see value in code review	NaN	Yes, it's part of our process	Not sure	I have little or no influence	C++;HTML/CSS;Java;JavaScript;Python;SQL;VBA	HTML/CSS;Java;JavaScript;SQL;WebAssembly	Couchbase;MongoDB;MySQL;Oracle;PostgreSQL;SQLite	Couchbase;Firebase;MongoDB;MySQL;Oracle;Postgr...	Android;Linux;MacOS;Slack;Windows	Android;Docker;Kubernetes;Linux;Slack	Django;Express;Flask;jQuery;React.js;Spring	Flask;jQuery;React.js;Spring	Cordova;Node.js	Apache Spark;Hadoop;Node.js;React Native	IntelliJ;Notepad++;Vim	Linux-based	Outside of work, for personal projects	Not at all	NaN	Yes	Also Yes	Yes	Facebook	In real life (in person)	Username	I don't remember	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy
5	6	I am not primarily a developer, but I write co...	Yes	Never	The quality of OSS and closed source software ...	Employed full-time	Canada	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Mathematics or statistics	Taken an online course in programming or softw...	NaN	Data or business analyst;Data scientist or mac...	13	15	3	Very satisfied	Slightly satisfied	Very confident	No	Yes	I am not interested in new job opportunities	1-2 years ago	Write any code;Complete a take-home project;In...	No	Financial performance or funding status of the...	I heard about a job opportunity (from a recrui...	CAD	Canadian dollar	40000.0	Monthly	366420.0	15.0	There's no schedule or spec; I work on what se...	NaN	A few days each month	Home	A little above average	No	NaN	Yes, it's not part of our process but the deve...	Not sure	I have little or no influence	Java;R;SQL	Python;Scala;SQL	MongoDB;PostgreSQL	PostgreSQL	Android;Google Cloud Platform;Linux;Windows	Android;Google Cloud Platform;Linux;Windows	NaN	NaN	Hadoop	Hadoop;Pandas;TensorFlow;Unity 3D	Android Studio;Eclipse;PyCharm;RStudio;Visual ...	Windows	I do not use containers	Not at all	NaN	No	Yes	No	YouTube	In real life (in person)	Login	2011	A few times per month or weekly	Find answers to specific questions	Less than once per week	Stack Overflow was slightly faster	60+ minutes	Yes	I have never participated in Q&A on Stack Over...	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	28.0	Man	No	Straight / Heterosexual	East Asian	No	Too long	Neither easy nor difficult
6	7	I am a developer by profession	No	Never	The quality of OSS and closed source software ...	Independent contractor, freelancer, or self-em...	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Another engineering discipline (ex. civil, ele...	Taught yourself a new language, framework, or ...	Just me - I am a freelancer, sole proprietor, ...	Designer;Developer, front-end	6	17	4	Slightly satisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	NA - I am an independent contractor or self em...	Write any code	No	Remote work options;Opportunities for professi...	NaN	USD	United States dollar	NaN	NaN	NaN	32.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Distrac...	All or almost all the time (I'm full-time remote)	Other place, such as a coworking space or cafe	Average	Yes, because I see value in code review	6.0	NaN	NaN	NaN	HTML/CSS;JavaScript	HTML/CSS;JavaScript	NaN	NaN	MacOS;Windows	MacOS;Windows	React.js	React.js	NaN	NaN	PHPStorm	Windows	I do not use containers	NaN	Useful for decentralized currency (i.e., Bitcoin)	No	Yes	No	YouTube	NaN	NaN	2014	Daily or almost daily	Find answers to specific questions	3-5 times per week	NaN	NaN	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Neutral	A lot more welcome now than last year	NaN	42.0	Man	No	Straight / Heterosexual	White or of European descent	Yes	Appropriate in length	Neither easy nor difficult
7	8	I code primarily as a hobby	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	NaN	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	NaN	Developer, back-end;Engineer, site reliability	8	16	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	Bash/Shell/PowerShell;C;C++;Elixir;Erlang;Go;P...	Cassandra;Elasticsearch;MongoDB;MySQL;Oracle;R...	Cassandra;DynamoDB;Elasticsearch;Firebase;Mong...	AWS;Docker;Heroku;Linux;MacOS;Slack	Android;Arduino;AWS;Docker;Google Cloud Platfo...	Express;Flask;React.js;Spring	Django;Express;Flask;React.js;Vue.js	Hadoop;Node.js;Pandas	Ansible;Apache Spark;Chef;Hadoop;Node.js;Panda...	Atom;IntelliJ;IPython / Jupyter;PyCharm;Visual...	Linux-based	Development;Testing;Production;Outside of work...	NaN	Useful across many domains and could change ma...	Yes	SIGH	Yes	YouTube	In real life (in person)	Handle	2012	A few times per week	Find answers to specific questions;Learn how t...	Less than once per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Yes, definitely	A lot more welcome now than last year	Tech articles written by other developers;Indu...	24.0	Man	No	Straight / Heterosexual	NaN	NaN	Appropriate in length	Neither easy nor difficult
8	9	I am a developer by profession	Yes	Once a month or more often	The quality of OSS and closed source software ...	Employed full-time	New Zealand	No	Some college/university study without earning ...	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10 to 19 employees	Database administrator;Developer, back-end;Dev...	12	11	4	Slightly satisfied	Slightly satisfied	Somewhat confident	No	Not sure	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Interview with people in peer r...	Yes	Financial performance or funding status of the...	I was preparing for a job search	NZD	New Zealand dollar	138000.0	Yearly	95179.0	32.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	Less than once per month / Never	Office	A little above average	Yes, because I see value in code review	12.0	Yes, it's not part of our process but the deve...	Not sure	I have some influence	Bash/Shell/PowerShell;C#;HTML/CSS;JavaScript;P...	Bash/Shell/PowerShell;C;HTML/CSS;JavaScript;Ru...	DynamoDB;PostgreSQL;SQLite	PostgreSQL;Redis;SQLite	AWS;Docker;Heroku;Linux;MacOS;Slack	AWS;Docker;Heroku;Linux;MacOS;Slack;Other(s):	Express;Ruby on Rails;Other(s):	Express;Ruby on Rails;Other(s):	Node.js;Unity 3D	Node.js	Vim	MacOS	Development;Testing;Production	Not at all	An irresponsible use of resources	No	SIGH	Yes	Twitter	In real life (in person)	Username	2013	Daily or almost daily	Find answers to specific questions;Contribute ...	3-5 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	NaN	23.0	Man	No	Bisexual	White or of European descent	No	Appropriate in length	Neither easy nor difficult
9	10	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	NaN	NaN	10,000 or more employees	Data or business analyst;Data scientist or mac...	12	20	10	Slightly dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	Yes	I’m not actively looking, but I am open to new...	3-4 years ago	NaN	No	Languages, frameworks, and other technologies ...	NaN	INR	Indian rupee	950000.0	Yearly	13293.0	70.0	There's no schedule or spec; I work on what se...	NaN	A few days each month	Home	Far above average	Yes, because I see value in code review	4.0	Yes, it's part of our process	NaN	NaN	C#;Go;JavaScript;Python;R;SQL	C#;Go;JavaScript;Kotlin;Python;R;SQL	Elasticsearch;MongoDB;Microsoft SQL Server;MyS...	Elasticsearch;MongoDB;Microsoft SQL Server	Linux;Windows	Android;Linux;Raspberry Pi;Windows	Angular/Angular.js;ASP.NET;Django;Express;Flas...	Angular/Angular.js;ASP.NET;Django;Express;Flas...	.NET;Node.js;Pandas;Torch/PyTorch	.NET;Node.js;TensorFlow;Torch/PyTorch	Android Studio;Eclipse;IPython / Jupyter;Notep...	Windows	NaN	Not at all	Useful for immutable record keeping outside of...	No	Yes	Yes	YouTube	Neither	Screen Name	NaN	Multiple times per day	Find answers to specific questions;Get a sense...	3-5 times per week	They were about the same	NaN	Yes	A few times per month or weekly	Yes	No, and I don't know what those are	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Tech...	NaN	NaN	NaN	NaN	NaN	Yes	Too long	Difficult
10	11	I code primarily as a hobby	Yes	Once a month or more often	The quality of OSS and closed source software ...	NaN	Antigua and Barbuda	Yes, full-time	Primary/elementary school	NaN	Contributed to open source software	NaN	NaN	2	11	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Other(s):	Other(s):	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Linux-based	NaN	NaN	Useful for immutable record keeping outside of...	No	Yes	NaN	NaN	Neither	Username	I don't remember	Less than once per month or monthly	Find answers to specific questions	Less than once per week	NaN	NaN	Yes	NaN	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not at all	Not applicable - I did not use Stack Overflow ...	NaN	22.0	Man	Yes	Bisexual;Gay or Lesbian;Straight / Heterosexual	Black or of African descent;East Asian;Hispani...	No	Appropriate in length	Easy
11	12	I am a student who is learning to code	No	Never	OSS is, on average, of HIGHER quality than pro...	Employed part-time	Canada	Yes, full-time	Some college/university study without earning ...	Mathematics or statistics	Taken an online course in programming or softw...	NaN	Data or business analyst;Data scientist or mac...	5	16	NaN	NaN	NaN	NaN	NaN	NaN	I am not interested in new job opportunities	Less than a year ago	NaN	NaN	Financial performance or funding status of the...	My job status changed (promotion, new job, etc.)	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;HTML/CSS;Java;Python;R;SQL	Bash/Shell/PowerShell;C++;Go;Python;R;Scala;SQL	MySQL;PostgreSQL;SQLite	Elasticsearch;MongoDB;MySQL;PostgreSQL	AWS;Docker;Google Cloud Platform;Linux;MacOS;S...	AWS;Linux;MacOS;Slack	NaN	NaN	Ansible;Chef;Hadoop;Pandas;TensorFlow	Ansible;Apache Spark;Chef;Hadoop;Pandas;Tensor...	IPython / Jupyter;PyCharm;RStudio;Sublime Text...	MacOS	Testing;Production	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2014	A few times per month or weekly	Find answers to specific questions;Learn how t...	1-2 times per week	The other resource was slightly faster	11-30 minutes	Not sure / can't remember	NaN	No, I knew that Stack Overflow had a job board...	Yes	Yes, somewhat	Just as welcome now as I felt last year	Courses on technologies you're interested in	21.0	Woman	No	Straight / Heterosexual	Black or of African descent	No	Appropriate in length	Easy

In [11]:

df.tail(12)

Out[11]:

	Respondent	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
88871	87386	NaN	Yes	Less than once a month but more than once per ...	OSS is, on average, of LOWER quality than prop...	Independent contractor, freelancer, or self-em...	Italy	Yes, full-time	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	NaN	Student	10	11	5	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Assembly;Bash/Shell/PowerShell;C;C#;HTML/CSS;J...	Assembly;C;Python	MariaDB;Microsoft SQL Server;MySQL;PostgreSQL	NaN	Android;Arduino;Docker;Heroku;Linux;WordPress	Google Cloud Platform	jQuery	Vue.js	Unity 3D	NaN	Emacs;Notepad++;Sublime Text;Visual Studio Code	Linux-based	I do not use containers	NaN	A passing fad	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2012	A few times per week	Find answers to specific questions;Learn how t...	1-2 times per week	Stack Overflow was much faster	11-30 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	No, not at all	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	21.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
88872	87739	NaN	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed part-time	Czech Republic	Yes, full-time	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	500 to 999 employees	Academic researcher;Designer;Developer, game o...	9	16	3	NaN	NaN	Somewhat confident	No	Not sure	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C;C++;HTML/CSS;JavaScript;PHP;Python;SQL	C++;HTML/CSS;PHP;Python	MySQL	MySQL	Android;Linux;WordPress	Android;Linux	jQuery	jQuery	TensorFlow	NaN	Vim	Linux-based	I do not use containers	NaN	A passing fad	No	Yes	Yes	YouTube	In real life (in person)	Login	2009	A few times per month or weekly	Find answers to specific questions	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	NaN	25.0	NaN	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
88873	88062	NaN	No	Never	OSS is, on average, of LOWER quality than prop...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88874	88076	NaN	No	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88875	88182	NaN	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed part-time	Pakistan	NaN	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	10 to 19 employees	Academic researcher	1	Younger than 5 years	Less than 1 year	NaN	NaN	Very confident	Not sure	Yes	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript	NaN	NaN	NaN	Google Cloud Platform	NaN	Other(s):	NaN	Other(s):	NaN	Android Studio	Windows	Development	NaN	Useful across many domains and could change ma...	Yes	Also Yes	What?	Twitter	Neither	Username	NaN	I have never visited Stack Overflow (before to...	Get a sense of belonging to the developer comm...	Less than once per week	The other resource was slightly faster	60+ minutes	NaN	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Not applicable - I did not use Stack Overflow ...	Courses on technologies you're interested in	NaN	Man	No	Straight / Heterosexual	NaN	Yes	Too short	Neither easy nor difficult
88876	88212	NaN	No	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Spain	No	Secondary school (e.g. American high school, G...	NaN	Taught yourself a new language, framework, or ...	10 to 19 employees	Designer;Developer, front-end;Developer, full-...	18	7	15	NaN	NaN	Somewhat confident	Yes	I am already a manager	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;JavaScript;Python	JavaScript	MySQL;PostgreSQL	PostgreSQL	NaN	Arduino	Django;React.js	Django;React.js	NaN	NaN	Atom;Visual Studio Code	MacOS	I do not use containers	Non-currency applications of blockchain	Useful for immutable record keeping outside of...	No	SIGH	No	WhatsApp	In real life (in person)	Username	2008	A few times per week	Find answers to specific questions	6-10 times per week	Stack Overflow was slightly faster	0-10 minutes	Yes	I have never participated in Q&A on Stack Over...	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	No, not at all	NaN	Tech articles written by other developers;Indu...	40.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
88877	88282	NaN	Yes	Once a month or more often	The quality of OSS and closed source software ...	Not employed, but looking for work	United States	No	Some college/university study without earning ...	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	NaN	Developer, back-end;Developer, desktop or ente...	38	10	38	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;Go;HTML/CSS;JavaScript;W...	Bash/Shell/PowerShell;C;Go;HTML/CSS;JavaScript...	NaN	NaN	Linux	Linux;Raspberry Pi	React.js	Vue.js	Node.js	Ansible	Vim	Linux-based	I do not use containers	NaN	An irresponsible use of resources	No	NaN	Yes	I don't use social media	In real life (in person)	Username	I don't remember	A few times per month or weekly	Find answers to specific questions	1-2 times per week	They were about the same	NaN	Yes	I have never participated in Q&A on Stack Over...	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	NaN	NaN	Man	No	Straight / Heterosexual	NaN	No	Too short	Neither easy nor difficult
88878	88377	NaN	Yes	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	Not employed, and not looking for work	Canada	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;JavaScript;Other(s):	C++;HTML/CSS;JavaScript;SQL;WebAssembly;Other(s):	Firebase;SQLite	Firebase;MySQL;SQLite	Linux	Google Cloud Platform;Linux	jQuery	jQuery;Vue.js	Node.js	React Native;Unity 3D;Unreal Engine	Atom;Visual Studio;Visual Studio Code	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	What?	YouTube	NaN	Username	I don't remember	A few times per week	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	I have never participated in Q&A on Stack Over...	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not at all	NaN	Tech articles written by other developers;Tech...	NaN	Man	No	NaN	NaN	No	Appropriate in length	Easy
88879	88601	NaN	No	Never	The quality of OSS and closed source software ...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88880	88802	NaN	No	Never	NaN	Employed full-time	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88881	88816	NaN	No	Never	OSS is, on average, of HIGHER quality than pro...	Independent contractor, freelancer, or self-em...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88882	88863	NaN	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, and not looking for work	Spain	Yes, full-time	Professional degree (JD, MD, etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	NaN	NaN	8	11	3	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;HTML/CSS;Java;JavaScript...	C;C++;C#;HTML/CSS;Java;Objective-C;SQL;Swift	DynamoDB;MariaDB;Microsoft SQL Server;MySQL	MariaDB;MySQL;Oracle;SQLite	iOS;Linux;MacOS;WordPress	Android;AWS;Google Cloud Platform;IBM Cloud or...	Django;jQuery;React.js	Django	Unity 3D;Unreal Engine	NaN	Android Studio;Atom;Eclipse;NetBeans;Notepad++...	Linux-based	Production	NaN	Useful for decentralized currency (i.e., Bitcoin)	Yes	Also Yes	Yes	WhatsApp	In real life (in person)	NaN	2012	Daily or almost daily	Find answers to specific questions;Learn how t...	6-10 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Indu...	18.0	Man	No	Straight / Heterosexual	Hispanic or Latino/Latina;White or of European...	No	Appropriate in length	Easy

Selecting rows and columns¶

In [12]:

people={"first":["Corey","John","Jane"],
        "last":["Schafer","Doe","Doe"],
        "email":["Corey@gmail.com","John@gmail.com","Jane@gmail.com"]}
ex_df = pd.DataFrame(people)

In [13]:

ex_df

Out[13]:

	first	last	email
0	Corey	Schafer	Corey@gmail.com
1	John	Doe	John@gmail.com
2	Jane	Doe	Jane@gmail.com

In [14]:

ex_df["email"]

Out[14]:

0    Corey@gmail.com
1     John@gmail.com
2     Jane@gmail.com
Name: email, dtype: object

In [15]:

ex_df[["last","email"]]

Out[15]:

	last	email
0	Schafer	Corey@gmail.com
1	Doe	John@gmail.com
2	Doe	Jane@gmail.com

In [16]:

ex_df.columns

Out[16]:

Index(['first', 'last', 'email'], dtype='object')

In [17]:

ex_df.iloc[0] #returns the row of specified index

Out[17]:

first              Corey
last             Schafer
email    Corey@gmail.com
Name: 0, dtype: object

In [18]:

ex_df.iloc[[0,1]] #returns the rows of specified indices

Out[18]:

	first	last	email
0	Corey	Schafer	Corey@gmail.com
1	John	Doe	John@gmail.com

In [19]:

ex_df.iloc[[0,1],2] #iloc can only use integer values and thus we cannot put the column names

Out[19]:

0    Corey@gmail.com
1     John@gmail.com
Name: email, dtype: object

In [20]:

ex_df.loc[0]

Out[20]:

first              Corey
last             Schafer
email    Corey@gmail.com
Name: 0, dtype: object

In [21]:

ex_df.loc[[0,1]] #loc is similar to iloc when using integers

Out[21]:

	first	last	email
0	Corey	Schafer	Corey@gmail.com
1	John	Doe	John@gmail.com

In [22]:

ex_df.loc[[0,1],["email","last"]] #loc can only use integer as well as string values and thus we can put the column names

Out[22]:

	email	last
0	Corey@gmail.com	Schafer
1	John@gmail.com	Doe

In [23]:

df.shape

Out[23]:

(88883, 85)

In [24]:

df.columns

Out[24]:

Index(['Respondent', 'MainBranch', 'Hobbyist', 'OpenSourcer', 'OpenSource',
       'Employment', 'Country', 'Student', 'EdLevel', 'UndergradMajor',
       'EduOther', 'OrgSize', 'DevType', 'YearsCode', 'Age1stCode',
       'YearsCodePro', 'CareerSat', 'JobSat', 'MgrIdiot', 'MgrMoney',
       'MgrWant', 'JobSeek', 'LastHireDate', 'LastInt', 'FizzBuzz',
       'JobFactors', 'ResumeUpdate', 'CurrencySymbol', 'CurrencyDesc',
       'CompTotal', 'CompFreq', 'ConvertedComp', 'WorkWeekHrs', 'WorkPlan',
       'WorkChallenge', 'WorkRemote', 'WorkLoc', 'ImpSyn', 'CodeRev',
       'CodeRevHrs', 'UnitTests', 'PurchaseHow', 'PurchaseWhat',
       'LanguageWorkedWith', 'LanguageDesireNextYear', 'DatabaseWorkedWith',
       'DatabaseDesireNextYear', 'PlatformWorkedWith',
       'PlatformDesireNextYear', 'WebFrameWorkedWith',
       'WebFrameDesireNextYear', 'MiscTechWorkedWith',
       'MiscTechDesireNextYear', 'DevEnviron', 'OpSys', 'Containers',
       'BlockchainOrg', 'BlockchainIs', 'BetterLife', 'ITperson', 'OffOn',
       'SocialMedia', 'Extraversion', 'ScreenName', 'SOVisit1st',
       'SOVisitFreq', 'SOVisitTo', 'SOFindAnswer', 'SOTimeSaved',
       'SOHowMuchTime', 'SOAccount', 'SOPartFreq', 'SOJobs', 'EntTeams',
       'SOComm', 'WelcomeChange', 'SONewContent', 'Age', 'Gender', 'Trans',
       'Sexuality', 'Ethnicity', 'Dependents', 'SurveyLength', 'SurveyEase'],
      dtype='object')

In [25]:

df["Hobbyist"]

Out[25]:

0        Yes
1         No
2        Yes
3         No
4        Yes
        ... 
88878    Yes
88879     No
88880     No
88881     No
88882    Yes
Name: Hobbyist, Length: 88883, dtype: object

In [26]:

df["Hobbyist"].value_counts()

Out[26]:

Yes    71257
No     17626
Name: Hobbyist, dtype: int64

In [27]:

df.loc[0]

Out[27]:

Respondent                                                                1
MainBranch                           I am a student who is learning to code
Hobbyist                                                                Yes
OpenSourcer                                                           Never
OpenSource                The quality of OSS and closed source software ...
Employment                           Not employed, and not looking for work
Country                                                      United Kingdom
Student                                                                  No
EdLevel                                           Primary/elementary school
UndergradMajor                                                          NaN
EduOther                  Taught yourself a new language, framework, or ...
OrgSize                                                                 NaN
DevType                                                                 NaN
YearsCode                                                                 4
Age1stCode                                                               10
YearsCodePro                                                            NaN
CareerSat                                                               NaN
JobSat                                                                  NaN
MgrIdiot                                                                NaN
MgrMoney                                                                NaN
MgrWant                                                                 NaN
JobSeek                                                                 NaN
LastHireDate                                                            NaN
LastInt                                                                 NaN
FizzBuzz                                                                NaN
JobFactors                                                              NaN
ResumeUpdate                                                            NaN
CurrencySymbol                                                          NaN
CurrencyDesc                                                            NaN
CompTotal                                                               NaN
CompFreq                                                                NaN
ConvertedComp                                                           NaN
WorkWeekHrs                                                             NaN
WorkPlan                                                                NaN
WorkChallenge                                                           NaN
WorkRemote                                                              NaN
WorkLoc                                                                 NaN
ImpSyn                                                                  NaN
CodeRev                                                                 NaN
CodeRevHrs                                                              NaN
UnitTests                                                               NaN
PurchaseHow                                                             NaN
PurchaseWhat                                                            NaN
LanguageWorkedWith                          HTML/CSS;Java;JavaScript;Python
LanguageDesireNextYear      C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL
DatabaseWorkedWith                                                   SQLite
DatabaseDesireNextYear                                                MySQL
PlatformWorkedWith                                            MacOS;Windows
PlatformDesireNextYear                              Android;Arduino;Windows
WebFrameWorkedWith                                             Django;Flask
WebFrameDesireNextYear                                         Flask;jQuery
MiscTechWorkedWith                                                  Node.js
MiscTechDesireNextYear                                              Node.js
DevEnviron                                       IntelliJ;Notepad++;PyCharm
OpSys                                                               Windows
Containers                                          I do not use containers
BlockchainOrg                                                           NaN
BlockchainIs                                                            NaN
BetterLife                                                              Yes
ITperson                           Fortunately, someone else has that title
OffOn                                                                   Yes
SocialMedia                                                         Twitter
Extraversion                                                         Online
ScreenName                                                         Username
SOVisit1st                                                             2017
SOVisitFreq                                 A few times per month or weekly
SOVisitTo                 Find answers to specific questions;Learn how t...
SOFindAnswer                                             3-5 times per week
SOTimeSaved                                  Stack Overflow was much faster
SOHowMuchTime                                                 31-60 minutes
SOAccount                                                                No
SOPartFreq                                                              NaN
SOJobs                    No, I didn't know that Stack Overflow had a jo...
EntTeams                                No, and I don't know what those are
SOComm                                                              Neutral
WelcomeChange                       Just as welcome now as I felt last year
SONewContent              Tech articles written by other developers;Indu...
Age                                                                      14
Gender                                                                  Man
Trans                                                                    No
Sexuality                                           Straight / Heterosexual
Ethnicity                                                               NaN
Dependents                                                               No
SurveyLength                                          Appropriate in length
SurveyEase                                       Neither easy nor difficult
Name: 0, dtype: object

In [28]:

df.loc[0,"Hobbyist"]

Out[28]:

'Yes'

In [29]:

df.loc[[0],["Hobbyist"]]

Out[29]:

	Hobbyist
0	Yes

In [30]:

df.loc[[0,1,5],"Hobbyist"]

Out[30]:

0    Yes
1     No
5    Yes
Name: Hobbyist, dtype: object

In [31]:

df.loc[[0,1,5],["Hobbyist"]]

Out[31]:

	Hobbyist
0	Yes
1	No
5	Yes

In [32]:

df.loc[0:5,"Hobbyist"]

Out[32]:

0    Yes
1     No
2    Yes
3     No
4    Yes
5    Yes
Name: Hobbyist, dtype: object

In [33]:

df.loc[0:5,"Hobbyist":"Employment"]

Out[33]:

	Hobbyist	OpenSourcer	OpenSource	Employment
0	Yes	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work
1	No	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work
2	Yes	Never	The quality of OSS and closed source software ...	Employed full-time
3	No	Never	The quality of OSS and closed source software ...	Employed full-time
4	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time
5	Yes	Never	The quality of OSS and closed source software ...	Employed full-time

How to set, reset and use indexes¶

In [34]:

ex_df.set_index("email")

Out[34]:

	first	last
email
Corey@gmail.com	Corey	Schafer
John@gmail.com	John	Doe
Jane@gmail.com	Jane	Doe

In [35]:

ex_df

Out[35]:

	first	last	email
0	Corey	Schafer	Corey@gmail.com
1	John	Doe	John@gmail.com
2	Jane	Doe	Jane@gmail.com

In [36]:

#dataframe did not change the index

In [37]:

ex_df.set_index("email",inplace=True)

In [38]:

ex_df

Out[38]:

	first	last
email
Corey@gmail.com	Corey	Schafer
John@gmail.com	John	Doe
Jane@gmail.com	Jane	Doe

In [39]:

#inplace will modify the dataframe

In [40]:

ex_df.index

Out[40]:

Index(['Corey@gmail.com', 'John@gmail.com', 'Jane@gmail.com'], dtype='object', name='email')

In [41]:

ex_df.loc["Corey@gmail.com"] #we can actually use strings to find rows

Out[41]:

first      Corey
last     Schafer
Name: Corey@gmail.com, dtype: object

In [42]:

ex_df.loc["Corey@gmail.com" , "last"] #we can actually use strings to find by rows and columns

Out[42]:

'Schafer'

In [43]:

# ex_df.loc[0] would throw an error until we reset back index to integers

In [44]:

ex_df.iloc[0] #this will still work

Out[44]:

first      Corey
last     Schafer
Name: Corey@gmail.com, dtype: object

In [45]:

ex_df.reset_index(inplace=True) #as the name suggests it resets to default index

In [46]:

ex_df

Out[46]:

	email	first	last
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	Jane@gmail.com	Jane	Doe

In [47]:

df = pd.read_csv("survey_results_public.csv" , index_col="Respondent") #we can change the index while reading
df.head()

Out[47]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
1	I am a student who is learning to code	Yes	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	4	10	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript;Python	C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL	SQLite	MySQL	MacOS;Windows	Android;Arduino;Windows	Django;Flask	Flask;jQuery	Node.js	Node.js	IntelliJ;Notepad++;PyCharm	Windows	I do not use containers	NaN	NaN	Yes	Fortunately, someone else has that title	Yes	Twitter	Online	Username	2017	A few times per month or weekly	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	31-60 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	I am a student who is learning to code	No	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Developer, desktop or enterprise applications;...	NaN	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Financial performance or funding status of the...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C++;HTML/CSS;Python	C++;HTML/CSS;JavaScript;SQL	NaN	MySQL	Windows	Windows	Django	Django	NaN	NaN	Atom;PyCharm	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	Yes	Instagram	Online	Username	2017	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
3	I am not primarily a developer, but I write co...	Yes	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	Taught yourself a new language, framework, or ...	100 to 499 employees	Designer;Developer, back-end;Developer, front-...	3	22	1	Slightly satisfied	Slightly satisfied	Not at all confident	Not sure	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	THB	Thai baht	23000.0	Monthly	8820.0	40.0	There's no schedule or spec; I work on what se...	Distracting work environment;Inadequate access...	Less than once per month / Never	Home	Average	No	NaN	No, but I think we should	Not sure	I have little or no influence	HTML/CSS	Elixir;HTML/CSS	PostgreSQL	PostgreSQL	NaN	NaN	NaN	Other(s):	NaN	NaN	Vim;Visual Studio Code	Linux-based	I do not use containers	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
4	I am a developer by profession	No	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
5	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Academic researcher;Developer, desktop or ente...	16	14	9	Very dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	No	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	UAH	Ukrainian hryvnia	NaN	NaN	NaN	55.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	A little above average	Yes, because I see value in code review	NaN	Yes, it's part of our process	Not sure	I have little or no influence	C++;HTML/CSS;Java;JavaScript;Python;SQL;VBA	HTML/CSS;Java;JavaScript;SQL;WebAssembly	Couchbase;MongoDB;MySQL;Oracle;PostgreSQL;SQLite	Couchbase;Firebase;MongoDB;MySQL;Oracle;Postgr...	Android;Linux;MacOS;Slack;Windows	Android;Docker;Kubernetes;Linux;Slack	Django;Express;Flask;jQuery;React.js;Spring	Flask;jQuery;React.js;Spring	Cordova;Node.js	Apache Spark;Hadoop;Node.js;React Native	IntelliJ;Notepad++;Vim	Linux-based	Outside of work, for personal projects	Not at all	NaN	Yes	Also Yes	Yes	Facebook	In real life (in person)	Username	I don't remember	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy

In [48]:

schema_df

Out[48]:

	Column	QuestionText
0	Respondent	Randomized respondent ID number (not in order ...
1	MainBranch	Which of the following options best describes ...
2	Hobbyist	Do you code as a hobby?
3	OpenSourcer	How often do you contribute to open source?
4	OpenSource	How do you feel about the quality of open sour...
5	Employment	Which of the following best describes your cur...
6	Country	In which country do you currently reside?
7	Student	Are you currently enrolled in a formal, degree...
8	EdLevel	Which of the following best describes the high...
9	UndergradMajor	What was your main or most important field of ...
10	EduOther	Which of the following types of non-degree edu...
11	OrgSize	Approximately how many people are employed by ...
12	DevType	Which of the following describe you? Please se...
13	YearsCode	Including any education, how many years have y...
14	Age1stCode	At what age did you write your first line of c...
15	YearsCodePro	How many years have you coded professionally (...
16	CareerSat	Overall, how satisfied are you with your caree...
17	JobSat	How satisfied are you with your current job? (...
18	MgrIdiot	How confident are you that your manager knows ...
19	MgrMoney	Do you believe that you need to be a manager t...
20	MgrWant	Do you want to become a manager yourself in th...
21	JobSeek	Which of the following best describes your cur...
22	LastHireDate	When was the last time that you took a job wit...
23	LastInt	In your most recent successful job interview (...
24	FizzBuzz	Have you ever been asked to solve FizzBuzz in ...
25	JobFactors	Imagine that you are deciding between two job ...
26	ResumeUpdate	Think back to the last time you updated your r...
27	CurrencySymbol	Which currency do you use day-to-day? If your ...
28	CurrencyDesc	Which currency do you use day-to-day? If your ...
29	CompTotal	What is your current total compensation (salar...
30	CompFreq	Is that compensation weekly, monthly, or yearly?
31	ConvertedComp	Salary converted to annual USD salaries using ...
32	WorkWeekHrs	On average, how many hours per week do you work?
33	WorkPlan	How structured or planned is your work?
34	WorkChallenge	Of these options, what are your greatest chall...
35	WorkRemote	How often do you work remotely?
36	WorkLoc	Where would you prefer to work?
37	ImpSyn	For the specific work you do, and the years of...
38	CodeRev	Do you review code as part of your work?
39	CodeRevHrs	On average, how many hours per week do you spe...
40	UnitTests	Does your company regularly employ unit tests ...
41	PurchaseHow	How does your company make decisions about pur...
42	PurchaseWhat	What level of influence do you, personally, ha...
43	LanguageWorkedWith	Which of the following programming, scripting,...
44	LanguageDesireNextYear	Which of the following programming, scripting,...
45	DatabaseWorkedWith	Which of the following database environments h...
46	DatabaseDesireNextYear	Which of the following database environments h...
47	PlatformWorkedWith	Which of the following platforms have you done...
48	PlatformDesireNextYear	Which of the following platforms have you done...
49	WebFrameWorkedWith	Which of the following web frameworks have you...
50	WebFrameDesireNextYear	Which of the following web frameworks have you...
51	MiscTechWorkedWith	Which of the following other frameworks, libra...
52	MiscTechDesireNextYear	Which of the following other frameworks, libra...
53	DevEnviron	Which development environment(s) do you use re...
54	OpSys	What is the primary operating system in which ...
55	Containers	How do you use containers (Docker, Open Contai...
56	BlockchainOrg	How is your organization thinking about or imp...
57	BlockchainIs	Blockchain / cryptocurrency technology is prim...
58	BetterLife	Do you think people born today will have a bet...
59	ITperson	Are you the "IT support person" for your family?
60	OffOn	Have you tried turning it off and on again?
61	SocialMedia	What social media site do you use the most?
62	Extraversion	Do you prefer online chat or IRL conversations?
63	ScreenName	What do you call it?
64	SOVisit1st	To the best of your memory, when did you first...
65	SOVisitFreq	How frequently would you say you visit Stack O...
66	SOVisitTo	I visit Stack Overflow to... (check all that a...
67	SOFindAnswer	On average, how many times a week do you find ...
68	SOTimeSaved	Think back to the last time you solved a codin...
69	SOHowMuchTime	About how much time did you save? If you're no...
70	SOAccount	Do you have a Stack Overflow account?
71	SOPartFreq	How frequently would you say you participate i...
72	SOJobs	Have you ever used or visited Stack Overflow J...
73	EntTeams	Have you ever used Stack Overflow for Enterpri...
74	SOComm	Do you consider yourself a member of the Stack...
75	WelcomeChange	Compared to last year, how welcome do you feel...
76	SONewContent	Would you like to see any of the following on ...
77	Age	What is your age (in years)? If you prefer not...
78	Gender	Which of the following do you currently identi...
79	Trans	Do you identify as transgender?
80	Sexuality	Which of the following do you currently identi...
81	Ethnicity	Which of the following do you identify as? Ple...
82	Dependents	Do you have any dependents (e.g., children, el...
83	SurveyLength	How do you feel about the length of the survey...
84	SurveyEase	How easy or difficult was this survey to compl...

In [49]:

schema_df = pd.read_csv("survey_results_schema.csv" , index_col="Column")
schema_df

Out[49]:

	QuestionText
Column
Respondent	Randomized respondent ID number (not in order ...
MainBranch	Which of the following options best describes ...
Hobbyist	Do you code as a hobby?
OpenSourcer	How often do you contribute to open source?
OpenSource	How do you feel about the quality of open sour...
Employment	Which of the following best describes your cur...
Country	In which country do you currently reside?
Student	Are you currently enrolled in a formal, degree...
EdLevel	Which of the following best describes the high...
UndergradMajor	What was your main or most important field of ...
EduOther	Which of the following types of non-degree edu...
OrgSize	Approximately how many people are employed by ...
DevType	Which of the following describe you? Please se...
YearsCode	Including any education, how many years have y...
Age1stCode	At what age did you write your first line of c...
YearsCodePro	How many years have you coded professionally (...
CareerSat	Overall, how satisfied are you with your caree...
JobSat	How satisfied are you with your current job? (...
MgrIdiot	How confident are you that your manager knows ...
MgrMoney	Do you believe that you need to be a manager t...
MgrWant	Do you want to become a manager yourself in th...
JobSeek	Which of the following best describes your cur...
LastHireDate	When was the last time that you took a job wit...
LastInt	In your most recent successful job interview (...
FizzBuzz	Have you ever been asked to solve FizzBuzz in ...
JobFactors	Imagine that you are deciding between two job ...
ResumeUpdate	Think back to the last time you updated your r...
CurrencySymbol	Which currency do you use day-to-day? If your ...
CurrencyDesc	Which currency do you use day-to-day? If your ...
CompTotal	What is your current total compensation (salar...
CompFreq	Is that compensation weekly, monthly, or yearly?
ConvertedComp	Salary converted to annual USD salaries using ...
WorkWeekHrs	On average, how many hours per week do you work?
WorkPlan	How structured or planned is your work?
WorkChallenge	Of these options, what are your greatest chall...
WorkRemote	How often do you work remotely?
WorkLoc	Where would you prefer to work?
ImpSyn	For the specific work you do, and the years of...
CodeRev	Do you review code as part of your work?
CodeRevHrs	On average, how many hours per week do you spe...
UnitTests	Does your company regularly employ unit tests ...
PurchaseHow	How does your company make decisions about pur...
PurchaseWhat	What level of influence do you, personally, ha...
LanguageWorkedWith	Which of the following programming, scripting,...
LanguageDesireNextYear	Which of the following programming, scripting,...
DatabaseWorkedWith	Which of the following database environments h...
DatabaseDesireNextYear	Which of the following database environments h...
PlatformWorkedWith	Which of the following platforms have you done...
PlatformDesireNextYear	Which of the following platforms have you done...
WebFrameWorkedWith	Which of the following web frameworks have you...
WebFrameDesireNextYear	Which of the following web frameworks have you...
MiscTechWorkedWith	Which of the following other frameworks, libra...
MiscTechDesireNextYear	Which of the following other frameworks, libra...
DevEnviron	Which development environment(s) do you use re...
OpSys	What is the primary operating system in which ...
Containers	How do you use containers (Docker, Open Contai...
BlockchainOrg	How is your organization thinking about or imp...
BlockchainIs	Blockchain / cryptocurrency technology is prim...
BetterLife	Do you think people born today will have a bet...
ITperson	Are you the "IT support person" for your family?
OffOn	Have you tried turning it off and on again?
SocialMedia	What social media site do you use the most?
Extraversion	Do you prefer online chat or IRL conversations?
ScreenName	What do you call it?
SOVisit1st	To the best of your memory, when did you first...
SOVisitFreq	How frequently would you say you visit Stack O...
SOVisitTo	I visit Stack Overflow to... (check all that a...
SOFindAnswer	On average, how many times a week do you find ...
SOTimeSaved	Think back to the last time you solved a codin...
SOHowMuchTime	About how much time did you save? If you're no...
SOAccount	Do you have a Stack Overflow account?
SOPartFreq	How frequently would you say you participate i...
SOJobs	Have you ever used or visited Stack Overflow J...
EntTeams	Have you ever used Stack Overflow for Enterpri...
SOComm	Do you consider yourself a member of the Stack...
WelcomeChange	Compared to last year, how welcome do you feel...
SONewContent	Would you like to see any of the following on ...
Age	What is your age (in years)? If you prefer not...
Gender	Which of the following do you currently identi...
Trans	Do you identify as transgender?
Sexuality	Which of the following do you currently identi...
Ethnicity	Which of the following do you identify as? Ple...
Dependents	Do you have any dependents (e.g., children, el...
SurveyLength	How do you feel about the length of the survey...
SurveyEase	How easy or difficult was this survey to compl...

In [50]:

schema_df.loc["Hobbyist"] #changing the index allowed us to easily search for the question associated with hobbyist column in df

Out[50]:

QuestionText    Do you code as a hobby?
Name: Hobbyist, dtype: object

In [51]:

schema_df.loc["MgrIdiot"] #but its's truncated

Out[51]:

QuestionText    How confident are you that your manager knows ...
Name: MgrIdiot, dtype: object

In [52]:

schema_df.loc["MgrIdiot","QuestionText"] #we got rid of the truncation without changing jupyter options

Out[52]:

'How confident are you that your manager knows what they’re doing?'

In [53]:

schema_df.sort_index() #sorts by index alphabetically

Out[53]:

	QuestionText
Column
Age	What is your age (in years)? If you prefer not...
Age1stCode	At what age did you write your first line of c...
BetterLife	Do you think people born today will have a bet...
BlockchainIs	Blockchain / cryptocurrency technology is prim...
BlockchainOrg	How is your organization thinking about or imp...
CareerSat	Overall, how satisfied are you with your caree...
CodeRev	Do you review code as part of your work?
CodeRevHrs	On average, how many hours per week do you spe...
CompFreq	Is that compensation weekly, monthly, or yearly?
CompTotal	What is your current total compensation (salar...
Containers	How do you use containers (Docker, Open Contai...
ConvertedComp	Salary converted to annual USD salaries using ...
Country	In which country do you currently reside?
CurrencyDesc	Which currency do you use day-to-day? If your ...
CurrencySymbol	Which currency do you use day-to-day? If your ...
DatabaseDesireNextYear	Which of the following database environments h...
DatabaseWorkedWith	Which of the following database environments h...
Dependents	Do you have any dependents (e.g., children, el...
DevEnviron	Which development environment(s) do you use re...
DevType	Which of the following describe you? Please se...
EdLevel	Which of the following best describes the high...
EduOther	Which of the following types of non-degree edu...
Employment	Which of the following best describes your cur...
EntTeams	Have you ever used Stack Overflow for Enterpri...
Ethnicity	Which of the following do you identify as? Ple...
Extraversion	Do you prefer online chat or IRL conversations?
FizzBuzz	Have you ever been asked to solve FizzBuzz in ...
Gender	Which of the following do you currently identi...
Hobbyist	Do you code as a hobby?
ITperson	Are you the "IT support person" for your family?
ImpSyn	For the specific work you do, and the years of...
JobFactors	Imagine that you are deciding between two job ...
JobSat	How satisfied are you with your current job? (...
JobSeek	Which of the following best describes your cur...
LanguageDesireNextYear	Which of the following programming, scripting,...
LanguageWorkedWith	Which of the following programming, scripting,...
LastHireDate	When was the last time that you took a job wit...
LastInt	In your most recent successful job interview (...
MainBranch	Which of the following options best describes ...
MgrIdiot	How confident are you that your manager knows ...
MgrMoney	Do you believe that you need to be a manager t...
MgrWant	Do you want to become a manager yourself in th...
MiscTechDesireNextYear	Which of the following other frameworks, libra...
MiscTechWorkedWith	Which of the following other frameworks, libra...
OffOn	Have you tried turning it off and on again?
OpSys	What is the primary operating system in which ...
OpenSource	How do you feel about the quality of open sour...
OpenSourcer	How often do you contribute to open source?
OrgSize	Approximately how many people are employed by ...
PlatformDesireNextYear	Which of the following platforms have you done...
PlatformWorkedWith	Which of the following platforms have you done...
PurchaseHow	How does your company make decisions about pur...
PurchaseWhat	What level of influence do you, personally, ha...
Respondent	Randomized respondent ID number (not in order ...
ResumeUpdate	Think back to the last time you updated your r...
SOAccount	Do you have a Stack Overflow account?
SOComm	Do you consider yourself a member of the Stack...
SOFindAnswer	On average, how many times a week do you find ...
SOHowMuchTime	About how much time did you save? If you're no...
SOJobs	Have you ever used or visited Stack Overflow J...
SONewContent	Would you like to see any of the following on ...
SOPartFreq	How frequently would you say you participate i...
SOTimeSaved	Think back to the last time you solved a codin...
SOVisit1st	To the best of your memory, when did you first...
SOVisitFreq	How frequently would you say you visit Stack O...
SOVisitTo	I visit Stack Overflow to... (check all that a...
ScreenName	What do you call it?
Sexuality	Which of the following do you currently identi...
SocialMedia	What social media site do you use the most?
Student	Are you currently enrolled in a formal, degree...
SurveyEase	How easy or difficult was this survey to compl...
SurveyLength	How do you feel about the length of the survey...
Trans	Do you identify as transgender?
UndergradMajor	What was your main or most important field of ...
UnitTests	Does your company regularly employ unit tests ...
WebFrameDesireNextYear	Which of the following web frameworks have you...
WebFrameWorkedWith	Which of the following web frameworks have you...
WelcomeChange	Compared to last year, how welcome do you feel...
WorkChallenge	Of these options, what are your greatest chall...
WorkLoc	Where would you prefer to work?
WorkPlan	How structured or planned is your work?
WorkRemote	How often do you work remotely?
WorkWeekHrs	On average, how many hours per week do you work?
YearsCode	Including any education, how many years have y...
YearsCodePro	How many years have you coded professionally (...

In [54]:

schema_df.sort_index(ascending=False) #sorts by index alphabetically in reverse order

Out[54]:

	QuestionText
Column
YearsCodePro	How many years have you coded professionally (...
YearsCode	Including any education, how many years have y...
WorkWeekHrs	On average, how many hours per week do you work?
WorkRemote	How often do you work remotely?
WorkPlan	How structured or planned is your work?
WorkLoc	Where would you prefer to work?
WorkChallenge	Of these options, what are your greatest chall...
WelcomeChange	Compared to last year, how welcome do you feel...
WebFrameWorkedWith	Which of the following web frameworks have you...
WebFrameDesireNextYear	Which of the following web frameworks have you...
UnitTests	Does your company regularly employ unit tests ...
UndergradMajor	What was your main or most important field of ...
Trans	Do you identify as transgender?
SurveyLength	How do you feel about the length of the survey...
SurveyEase	How easy or difficult was this survey to compl...
Student	Are you currently enrolled in a formal, degree...
SocialMedia	What social media site do you use the most?
Sexuality	Which of the following do you currently identi...
ScreenName	What do you call it?
SOVisitTo	I visit Stack Overflow to... (check all that a...
SOVisitFreq	How frequently would you say you visit Stack O...
SOVisit1st	To the best of your memory, when did you first...
SOTimeSaved	Think back to the last time you solved a codin...
SOPartFreq	How frequently would you say you participate i...
SONewContent	Would you like to see any of the following on ...
SOJobs	Have you ever used or visited Stack Overflow J...
SOHowMuchTime	About how much time did you save? If you're no...
SOFindAnswer	On average, how many times a week do you find ...
SOComm	Do you consider yourself a member of the Stack...
SOAccount	Do you have a Stack Overflow account?
ResumeUpdate	Think back to the last time you updated your r...
Respondent	Randomized respondent ID number (not in order ...
PurchaseWhat	What level of influence do you, personally, ha...
PurchaseHow	How does your company make decisions about pur...
PlatformWorkedWith	Which of the following platforms have you done...
PlatformDesireNextYear	Which of the following platforms have you done...
OrgSize	Approximately how many people are employed by ...
OpenSourcer	How often do you contribute to open source?
OpenSource	How do you feel about the quality of open sour...
OpSys	What is the primary operating system in which ...
OffOn	Have you tried turning it off and on again?
MiscTechWorkedWith	Which of the following other frameworks, libra...
MiscTechDesireNextYear	Which of the following other frameworks, libra...
MgrWant	Do you want to become a manager yourself in th...
MgrMoney	Do you believe that you need to be a manager t...
MgrIdiot	How confident are you that your manager knows ...
MainBranch	Which of the following options best describes ...
LastInt	In your most recent successful job interview (...
LastHireDate	When was the last time that you took a job wit...
LanguageWorkedWith	Which of the following programming, scripting,...
LanguageDesireNextYear	Which of the following programming, scripting,...
JobSeek	Which of the following best describes your cur...
JobSat	How satisfied are you with your current job? (...
JobFactors	Imagine that you are deciding between two job ...
ImpSyn	For the specific work you do, and the years of...
ITperson	Are you the "IT support person" for your family?
Hobbyist	Do you code as a hobby?
Gender	Which of the following do you currently identi...
FizzBuzz	Have you ever been asked to solve FizzBuzz in ...
Extraversion	Do you prefer online chat or IRL conversations?
Ethnicity	Which of the following do you identify as? Ple...
EntTeams	Have you ever used Stack Overflow for Enterpri...
Employment	Which of the following best describes your cur...
EduOther	Which of the following types of non-degree edu...
EdLevel	Which of the following best describes the high...
DevType	Which of the following describe you? Please se...
DevEnviron	Which development environment(s) do you use re...
Dependents	Do you have any dependents (e.g., children, el...
DatabaseWorkedWith	Which of the following database environments h...
DatabaseDesireNextYear	Which of the following database environments h...
CurrencySymbol	Which currency do you use day-to-day? If your ...
CurrencyDesc	Which currency do you use day-to-day? If your ...
Country	In which country do you currently reside?
ConvertedComp	Salary converted to annual USD salaries using ...
Containers	How do you use containers (Docker, Open Contai...
CompTotal	What is your current total compensation (salar...
CompFreq	Is that compensation weekly, monthly, or yearly?
CodeRevHrs	On average, how many hours per week do you spe...
CodeRev	Do you review code as part of your work?
CareerSat	Overall, how satisfied are you with your caree...
BlockchainOrg	How is your organization thinking about or imp...
BlockchainIs	Blockchain / cryptocurrency technology is prim...
BetterLife	Do you think people born today will have a bet...
Age1stCode	At what age did you write your first line of c...
Age	What is your age (in years)? If you prefer not...

In [55]:

schema_df #once again it didn't get modified

Out[55]:

	QuestionText
Column
Respondent	Randomized respondent ID number (not in order ...
MainBranch	Which of the following options best describes ...
Hobbyist	Do you code as a hobby?
OpenSourcer	How often do you contribute to open source?
OpenSource	How do you feel about the quality of open sour...
Employment	Which of the following best describes your cur...
Country	In which country do you currently reside?
Student	Are you currently enrolled in a formal, degree...
EdLevel	Which of the following best describes the high...
UndergradMajor	What was your main or most important field of ...
EduOther	Which of the following types of non-degree edu...
OrgSize	Approximately how many people are employed by ...
DevType	Which of the following describe you? Please se...
YearsCode	Including any education, how many years have y...
Age1stCode	At what age did you write your first line of c...
YearsCodePro	How many years have you coded professionally (...
CareerSat	Overall, how satisfied are you with your caree...
JobSat	How satisfied are you with your current job? (...
MgrIdiot	How confident are you that your manager knows ...
MgrMoney	Do you believe that you need to be a manager t...
MgrWant	Do you want to become a manager yourself in th...
JobSeek	Which of the following best describes your cur...
LastHireDate	When was the last time that you took a job wit...
LastInt	In your most recent successful job interview (...
FizzBuzz	Have you ever been asked to solve FizzBuzz in ...
JobFactors	Imagine that you are deciding between two job ...
ResumeUpdate	Think back to the last time you updated your r...
CurrencySymbol	Which currency do you use day-to-day? If your ...
CurrencyDesc	Which currency do you use day-to-day? If your ...
CompTotal	What is your current total compensation (salar...
CompFreq	Is that compensation weekly, monthly, or yearly?
ConvertedComp	Salary converted to annual USD salaries using ...
WorkWeekHrs	On average, how many hours per week do you work?
WorkPlan	How structured or planned is your work?
WorkChallenge	Of these options, what are your greatest chall...
WorkRemote	How often do you work remotely?
WorkLoc	Where would you prefer to work?
ImpSyn	For the specific work you do, and the years of...
CodeRev	Do you review code as part of your work?
CodeRevHrs	On average, how many hours per week do you spe...
UnitTests	Does your company regularly employ unit tests ...
PurchaseHow	How does your company make decisions about pur...
PurchaseWhat	What level of influence do you, personally, ha...
LanguageWorkedWith	Which of the following programming, scripting,...
LanguageDesireNextYear	Which of the following programming, scripting,...
DatabaseWorkedWith	Which of the following database environments h...
DatabaseDesireNextYear	Which of the following database environments h...
PlatformWorkedWith	Which of the following platforms have you done...
PlatformDesireNextYear	Which of the following platforms have you done...
WebFrameWorkedWith	Which of the following web frameworks have you...
WebFrameDesireNextYear	Which of the following web frameworks have you...
MiscTechWorkedWith	Which of the following other frameworks, libra...
MiscTechDesireNextYear	Which of the following other frameworks, libra...
DevEnviron	Which development environment(s) do you use re...
OpSys	What is the primary operating system in which ...
Containers	How do you use containers (Docker, Open Contai...
BlockchainOrg	How is your organization thinking about or imp...
BlockchainIs	Blockchain / cryptocurrency technology is prim...
BetterLife	Do you think people born today will have a bet...
ITperson	Are you the "IT support person" for your family?
OffOn	Have you tried turning it off and on again?
SocialMedia	What social media site do you use the most?
Extraversion	Do you prefer online chat or IRL conversations?
ScreenName	What do you call it?
SOVisit1st	To the best of your memory, when did you first...
SOVisitFreq	How frequently would you say you visit Stack O...
SOVisitTo	I visit Stack Overflow to... (check all that a...
SOFindAnswer	On average, how many times a week do you find ...
SOTimeSaved	Think back to the last time you solved a codin...
SOHowMuchTime	About how much time did you save? If you're no...
SOAccount	Do you have a Stack Overflow account?
SOPartFreq	How frequently would you say you participate i...
SOJobs	Have you ever used or visited Stack Overflow J...
EntTeams	Have you ever used Stack Overflow for Enterpri...
SOComm	Do you consider yourself a member of the Stack...
WelcomeChange	Compared to last year, how welcome do you feel...
SONewContent	Would you like to see any of the following on ...
Age	What is your age (in years)? If you prefer not...
Gender	Which of the following do you currently identi...
Trans	Do you identify as transgender?
Sexuality	Which of the following do you currently identi...
Ethnicity	Which of the following do you identify as? Ple...
Dependents	Do you have any dependents (e.g., children, el...
SurveyLength	How do you feel about the length of the survey...
SurveyEase	How easy or difficult was this survey to compl...

In [56]:

schema_df.sort_index(inplace=True) #sorts by index alphabetically
schema_df

Out[56]:

	QuestionText
Column
Age	What is your age (in years)? If you prefer not...
Age1stCode	At what age did you write your first line of c...
BetterLife	Do you think people born today will have a bet...
BlockchainIs	Blockchain / cryptocurrency technology is prim...
BlockchainOrg	How is your organization thinking about or imp...
CareerSat	Overall, how satisfied are you with your caree...
CodeRev	Do you review code as part of your work?
CodeRevHrs	On average, how many hours per week do you spe...
CompFreq	Is that compensation weekly, monthly, or yearly?
CompTotal	What is your current total compensation (salar...
Containers	How do you use containers (Docker, Open Contai...
ConvertedComp	Salary converted to annual USD salaries using ...
Country	In which country do you currently reside?
CurrencyDesc	Which currency do you use day-to-day? If your ...
CurrencySymbol	Which currency do you use day-to-day? If your ...
DatabaseDesireNextYear	Which of the following database environments h...
DatabaseWorkedWith	Which of the following database environments h...
Dependents	Do you have any dependents (e.g., children, el...
DevEnviron	Which development environment(s) do you use re...
DevType	Which of the following describe you? Please se...
EdLevel	Which of the following best describes the high...
EduOther	Which of the following types of non-degree edu...
Employment	Which of the following best describes your cur...
EntTeams	Have you ever used Stack Overflow for Enterpri...
Ethnicity	Which of the following do you identify as? Ple...
Extraversion	Do you prefer online chat or IRL conversations?
FizzBuzz	Have you ever been asked to solve FizzBuzz in ...
Gender	Which of the following do you currently identi...
Hobbyist	Do you code as a hobby?
ITperson	Are you the "IT support person" for your family?
ImpSyn	For the specific work you do, and the years of...
JobFactors	Imagine that you are deciding between two job ...
JobSat	How satisfied are you with your current job? (...
JobSeek	Which of the following best describes your cur...
LanguageDesireNextYear	Which of the following programming, scripting,...
LanguageWorkedWith	Which of the following programming, scripting,...
LastHireDate	When was the last time that you took a job wit...
LastInt	In your most recent successful job interview (...
MainBranch	Which of the following options best describes ...
MgrIdiot	How confident are you that your manager knows ...
MgrMoney	Do you believe that you need to be a manager t...
MgrWant	Do you want to become a manager yourself in th...
MiscTechDesireNextYear	Which of the following other frameworks, libra...
MiscTechWorkedWith	Which of the following other frameworks, libra...
OffOn	Have you tried turning it off and on again?
OpSys	What is the primary operating system in which ...
OpenSource	How do you feel about the quality of open sour...
OpenSourcer	How often do you contribute to open source?
OrgSize	Approximately how many people are employed by ...
PlatformDesireNextYear	Which of the following platforms have you done...
PlatformWorkedWith	Which of the following platforms have you done...
PurchaseHow	How does your company make decisions about pur...
PurchaseWhat	What level of influence do you, personally, ha...
Respondent	Randomized respondent ID number (not in order ...
ResumeUpdate	Think back to the last time you updated your r...
SOAccount	Do you have a Stack Overflow account?
SOComm	Do you consider yourself a member of the Stack...
SOFindAnswer	On average, how many times a week do you find ...
SOHowMuchTime	About how much time did you save? If you're no...
SOJobs	Have you ever used or visited Stack Overflow J...
SONewContent	Would you like to see any of the following on ...
SOPartFreq	How frequently would you say you participate i...
SOTimeSaved	Think back to the last time you solved a codin...
SOVisit1st	To the best of your memory, when did you first...
SOVisitFreq	How frequently would you say you visit Stack O...
SOVisitTo	I visit Stack Overflow to... (check all that a...
ScreenName	What do you call it?
Sexuality	Which of the following do you currently identi...
SocialMedia	What social media site do you use the most?
Student	Are you currently enrolled in a formal, degree...
SurveyEase	How easy or difficult was this survey to compl...
SurveyLength	How do you feel about the length of the survey...
Trans	Do you identify as transgender?
UndergradMajor	What was your main or most important field of ...
UnitTests	Does your company regularly employ unit tests ...
WebFrameDesireNextYear	Which of the following web frameworks have you...
WebFrameWorkedWith	Which of the following web frameworks have you...
WelcomeChange	Compared to last year, how welcome do you feel...
WorkChallenge	Of these options, what are your greatest chall...
WorkLoc	Where would you prefer to work?
WorkPlan	How structured or planned is your work?
WorkRemote	How often do you work remotely?
WorkWeekHrs	On average, how many hours per week do you work?
YearsCode	Including any education, how many years have y...
YearsCodePro	How many years have you coded professionally (...

Using conditionals to filter rows and columns¶

In [57]:

ex_df["last"]=="Doe"

Out[57]:

0    False
1     True
2     True
Name: last, dtype: bool

In [58]:

filt = ex_df["last"]=="Doe"

In [59]:

ex_df[filt]

Out[59]:

	email	first	last
1	John@gmail.com	John	Doe
2	Jane@gmail.com	Jane	Doe

In [60]:

ex_df.loc[filt] #this is preferred as we can also pass a column we want to see specifically

Out[60]:

	email	first	last
1	John@gmail.com	John	Doe
2	Jane@gmail.com	Jane	Doe

In [61]:

ex_df.loc[filt,"email"]

Out[61]:

1    John@gmail.com
2    Jane@gmail.com
Name: email, dtype: object

In [62]:

filt = (ex_df["last"]=="Doe") & (ex_df["first"]=="John")

In [63]:

ex_df.loc[filt,"email"]

Out[63]:

1    John@gmail.com
Name: email, dtype: object

In [64]:

filt = (ex_df["first"]=="John") | (ex_df["last"]=="Schafer")

In [65]:

ex_df.loc[filt,"email"]

Out[65]:

0    Corey@gmail.com
1     John@gmail.com
Name: email, dtype: object

In [66]:

ex_df.loc[~filt,"email"] #if we want neither first name John or last name Schafer

Out[66]:

2    Jane@gmail.com
Name: email, dtype: object

In [67]:

high_salary = (df["ConvertedComp"]>70000)

In [68]:

df.loc[high_salary,["Country","LanguageWorkedWith","ConvertedComp"]]

Out[68]:

	Country	LanguageWorkedWith	ConvertedComp
Respondent
6	Canada	Java;R;SQL	366420.0
9	New Zealand	Bash/Shell/PowerShell;C#;HTML/CSS;JavaScript;P...	95179.0
13	United States	Bash/Shell/PowerShell;HTML/CSS;JavaScript;PHP;...	90000.0
16	United Kingdom	Bash/Shell/PowerShell;C#;HTML/CSS;JavaScript;T...	455352.0
22	United States	Bash/Shell/PowerShell;C++;HTML/CSS;JavaScript;...	103000.0
...	...	...	...
88876	United States	Bash/Shell/PowerShell;C#;HTML/CSS;Java;Python;...	180000.0
88877	United States	Bash/Shell/PowerShell;C;Clojure;HTML/CSS;Java;...	2000000.0
88878	United States	HTML/CSS;JavaScript;Scala;TypeScript	130000.0
88879	Finland	Bash/Shell/PowerShell;C++;Python	82488.0
88882	Netherlands	C#;HTML/CSS;Java;JavaScript;PHP;Python	588012.0

22289 rows × 3 columns

In [69]:

countries = ["India","United Kingdom","United States","Canada","Germany"]
filt = (df["Country"]).isin(countries)&(high_salary)
df.loc[filt,["Country","ConvertedComp"]]

Out[69]:

	Country	ConvertedComp
Respondent
6	Canada	366420.0
13	United States	90000.0
16	United Kingdom	455352.0
22	United States	103000.0
26	United States	114000.0
...	...	...
88873	Canada	72522.0
88874	United States	2000000.0
88876	United States	180000.0
88877	United States	2000000.0
88878	United States	130000.0

16857 rows × 2 columns

In [70]:

filt = df["LanguageWorkedWith"].str.contains("Python",na=False)&(df["Country"]=="India") #NaN shouldn't be considered so we use na=False
df.loc[filt,"ConvertedComp"].mean()

Out[70]:

37093.13056835638

Updating rows and columns¶

In [71]:

ex_df.columns

Out[71]:

Index(['email', 'first', 'last'], dtype='object')

In [72]:

ex_df.columns=['email',"first_name","last_name"] #not  recommended as we need to type all columns

In [73]:

ex_df.columns

Out[73]:

Index(['email', 'first_name', 'last_name'], dtype='object')

In [74]:

ex_df

Out[74]:

	email	first_name	last_name
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	Jane@gmail.com	Jane	Doe

In [75]:

ex_df.columns = [ x.upper() for x in ex_df.columns]
ex_df

Out[75]:

	EMAIL	FIRST_NAME	LAST_NAME
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	Jane@gmail.com	Jane	Doe

In [76]:

ex_df.columns = ex_df.columns.str.replace("_"," ")
ex_df

Out[76]:

	EMAIL	FIRST NAME	LAST NAME
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	Jane@gmail.com	Jane	Doe

In [77]:

ex_df.rename(columns={"EMAIL":"email", "FIRST NAME":"first", "LAST NAME":"last"},inplace=True)
ex_df

Out[77]:

	email	first	last
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	Jane@gmail.com	Jane	Doe

In [78]:

ex_df.loc[2]=["JaneDoe@gmail.com","Jane","Doe"] #just for changing a single value we had to type the entire row so its not useful
ex_df

Out[78]:

	email	first	last
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	JaneDoe@gmail.com	Jane	Doe

In [79]:

ex_df.loc[2,["email","last"]]=["JaneSmith@gmail.com","Smith"]
ex_df

Out[79]:

	email	first	last
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	JaneSmith@gmail.com	Jane	Smith

In [80]:

filt = ex_df["email"]=="John@gmail.com"
ex_df[filt]["last"]="Smith" #we are trying to modifty the slice and not the actual dataframe
ex_df

<ipython-input-80-e049e2ca2e40>:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  ex_df[filt]["last"]="Smith" #we are trying to modifty the slice and not the actual dataframe

Out[80]:

	email	first	last
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Doe
2	JaneSmith@gmail.com	Jane	Smith

In [81]:

filt = ex_df["email"]=="John@gmail.com"
ex_df.loc[filt,"last"]="Smith"
ex_df

Out[81]:

	email	first	last
0	Corey@gmail.com	Corey	Schafer
1	John@gmail.com	John	Smith
2	JaneSmith@gmail.com	Jane	Smith

In [82]:

ex_df["email"] = ex_df["email"].str.lower()
ex_df

Out[82]:

	email	first	last
0	corey@gmail.com	Corey	Schafer
1	john@gmail.com	John	Smith
2	janesmith@gmail.com	Jane	Smith

In [83]:

ex_df["email"].apply(len)

Out[83]:

0    15
1    14
2    19
Name: email, dtype: int64

In [84]:

def update_email(email):
    return email.upper()
ex_df["email"].apply(update_email)

Out[84]:

0        COREY@GMAIL.COM
1         JOHN@GMAIL.COM
2    JANESMITH@GMAIL.COM
Name: email, dtype: object

In [85]:

ex_df["email"] = ex_df["email"].apply(update_email)
ex_df

Out[85]:

	email	first	last
0	COREY@GMAIL.COM	Corey	Schafer
1	JOHN@GMAIL.COM	John	Smith
2	JANESMITH@GMAIL.COM	Jane	Smith

In [86]:

ex_df["email"] = ex_df["email"].apply(lambda x: x.lower())
ex_df

Out[86]:

	email	first	last
0	corey@gmail.com	Corey	Schafer
1	john@gmail.com	John	Smith
2	janesmith@gmail.com	Jane	Smith

In [87]:

ex_df.apply(len) #length function applies length to each series in dataframe / counts rows in each column

Out[87]:

email    3
first    3
last     3
dtype: int64

In [88]:

ex_df.apply(len,axis="columns") #counts columns in each row

Out[88]:

0    3
1    3
2    3
dtype: int64

In [89]:

ex_df.apply(pd.Series.min)

Out[89]:

email    corey@gmail.com
first              Corey
last             Schafer
dtype: object

In [90]:

ex_df.apply(lambda x: x.min())

Out[90]:

email    corey@gmail.com
first              Corey
last             Schafer
dtype: object

In [91]:

#applymap only works on dataframes

In [92]:

ex_df.applymap(len) #it applies len function to each individual element of the dataframe

Out[92]:

	email	first	last
0	15	5	7
1	14	4	5
2	19	4	5

In [93]:

ex_df = ex_df.applymap(str.lower)
ex_df

Out[93]:

	email	first	last
0	corey@gmail.com	corey	schafer
1	john@gmail.com	john	smith
2	janesmith@gmail.com	jane	smith

In [94]:

ex_df["first"].map({"corey":"Chris", "jane":"Mary"}) #only when we need one-one maps. since we did not create a map for "john" we get back NaN

Out[94]:

0    Chris
1      NaN
2     Mary
Name: first, dtype: object

In [95]:

ex_df["first"].replace({"corey":"Chris", "jane":"Mary"})

Out[95]:

0    Chris
1     john
2     Mary
Name: first, dtype: object

In [96]:

df.rename(columns={"ConvertedComp":"SalaryUSD"},inplace=True)
df

Out[96]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
1	I am a student who is learning to code	Yes	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	4	10	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript;Python	C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL	SQLite	MySQL	MacOS;Windows	Android;Arduino;Windows	Django;Flask	Flask;jQuery	Node.js	Node.js	IntelliJ;Notepad++;PyCharm	Windows	I do not use containers	NaN	NaN	Yes	Fortunately, someone else has that title	Yes	Twitter	Online	Username	2017	A few times per month or weekly	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	31-60 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	I am a student who is learning to code	No	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Developer, desktop or enterprise applications;...	NaN	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Financial performance or funding status of the...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C++;HTML/CSS;Python	C++;HTML/CSS;JavaScript;SQL	NaN	MySQL	Windows	Windows	Django	Django	NaN	NaN	Atom;PyCharm	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	Yes	Instagram	Online	Username	2017	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
3	I am not primarily a developer, but I write co...	Yes	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	Taught yourself a new language, framework, or ...	100 to 499 employees	Designer;Developer, back-end;Developer, front-...	3	22	1	Slightly satisfied	Slightly satisfied	Not at all confident	Not sure	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	THB	Thai baht	23000.0	Monthly	8820.0	40.0	There's no schedule or spec; I work on what se...	Distracting work environment;Inadequate access...	Less than once per month / Never	Home	Average	No	NaN	No, but I think we should	Not sure	I have little or no influence	HTML/CSS	Elixir;HTML/CSS	PostgreSQL	PostgreSQL	NaN	NaN	NaN	Other(s):	NaN	NaN	Vim;Visual Studio Code	Linux-based	I do not use containers	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
4	I am a developer by profession	No	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
5	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Academic researcher;Developer, desktop or ente...	16	14	9	Very dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	No	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	UAH	Ukrainian hryvnia	NaN	NaN	NaN	55.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	A little above average	Yes, because I see value in code review	NaN	Yes, it's part of our process	Not sure	I have little or no influence	C++;HTML/CSS;Java;JavaScript;Python;SQL;VBA	HTML/CSS;Java;JavaScript;SQL;WebAssembly	Couchbase;MongoDB;MySQL;Oracle;PostgreSQL;SQLite	Couchbase;Firebase;MongoDB;MySQL;Oracle;Postgr...	Android;Linux;MacOS;Slack;Windows	Android;Docker;Kubernetes;Linux;Slack	Django;Express;Flask;jQuery;React.js;Spring	Flask;jQuery;React.js;Spring	Cordova;Node.js	Apache Spark;Hadoop;Node.js;React Native	IntelliJ;Notepad++;Vim	Linux-based	Outside of work, for personal projects	Not at all	NaN	Yes	Also Yes	Yes	Facebook	In real life (in person)	Username	I don't remember	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88377	NaN	Yes	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	Not employed, and not looking for work	Canada	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;JavaScript;Other(s):	C++;HTML/CSS;JavaScript;SQL;WebAssembly;Other(s):	Firebase;SQLite	Firebase;MySQL;SQLite	Linux	Google Cloud Platform;Linux	jQuery	jQuery;Vue.js	Node.js	React Native;Unity 3D;Unreal Engine	Atom;Visual Studio;Visual Studio Code	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	What?	YouTube	NaN	Username	I don't remember	A few times per week	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	I have never participated in Q&A on Stack Over...	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not at all	NaN	Tech articles written by other developers;Tech...	NaN	Man	No	NaN	NaN	No	Appropriate in length	Easy
88601	NaN	No	Never	The quality of OSS and closed source software ...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88802	NaN	No	Never	NaN	Employed full-time	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88816	NaN	No	Never	OSS is, on average, of HIGHER quality than pro...	Independent contractor, freelancer, or self-em...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88863	NaN	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, and not looking for work	Spain	Yes, full-time	Professional degree (JD, MD, etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	NaN	NaN	8	11	3	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;HTML/CSS;Java;JavaScript...	C;C++;C#;HTML/CSS;Java;Objective-C;SQL;Swift	DynamoDB;MariaDB;Microsoft SQL Server;MySQL	MariaDB;MySQL;Oracle;SQLite	iOS;Linux;MacOS;WordPress	Android;AWS;Google Cloud Platform;IBM Cloud or...	Django;jQuery;React.js	Django	Unity 3D;Unreal Engine	NaN	Android Studio;Atom;Eclipse;NetBeans;Notepad++...	Linux-based	Production	NaN	Useful for decentralized currency (i.e., Bitcoin)	Yes	Also Yes	Yes	WhatsApp	In real life (in person)	NaN	2012	Daily or almost daily	Find answers to specific questions;Learn how t...	6-10 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Indu...	18.0	Man	No	Straight / Heterosexual	Hispanic or Latino/Latina;White or of European...	No	Appropriate in length	Easy

88883 rows × 84 columns

In [97]:

df["Hobbyist"].map({"Yes":True,"No":False})

Out[97]:

Respondent
1         True
2        False
3         True
4        False
5         True
         ...  
88377     True
88601    False
88802    False
88816    False
88863     True
Name: Hobbyist, Length: 88883, dtype: bool

In [98]:

df["Hobbyist"] = df["Hobbyist"].map({"Yes":True,"No":False}) #this worked here as there was only Yes or No. if there was more we would have to use replace
df

Out[98]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
1	I am a student who is learning to code	True	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	4	10	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript;Python	C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL	SQLite	MySQL	MacOS;Windows	Android;Arduino;Windows	Django;Flask	Flask;jQuery	Node.js	Node.js	IntelliJ;Notepad++;PyCharm	Windows	I do not use containers	NaN	NaN	Yes	Fortunately, someone else has that title	Yes	Twitter	Online	Username	2017	A few times per month or weekly	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	31-60 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	I am a student who is learning to code	False	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Developer, desktop or enterprise applications;...	NaN	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Financial performance or funding status of the...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C++;HTML/CSS;Python	C++;HTML/CSS;JavaScript;SQL	NaN	MySQL	Windows	Windows	Django	Django	NaN	NaN	Atom;PyCharm	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	Yes	Instagram	Online	Username	2017	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
3	I am not primarily a developer, but I write co...	True	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	Taught yourself a new language, framework, or ...	100 to 499 employees	Designer;Developer, back-end;Developer, front-...	3	22	1	Slightly satisfied	Slightly satisfied	Not at all confident	Not sure	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	THB	Thai baht	23000.0	Monthly	8820.0	40.0	There's no schedule or spec; I work on what se...	Distracting work environment;Inadequate access...	Less than once per month / Never	Home	Average	No	NaN	No, but I think we should	Not sure	I have little or no influence	HTML/CSS	Elixir;HTML/CSS	PostgreSQL	PostgreSQL	NaN	NaN	NaN	Other(s):	NaN	NaN	Vim;Visual Studio Code	Linux-based	I do not use containers	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
4	I am a developer by profession	False	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
5	I am a developer by profession	True	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Academic researcher;Developer, desktop or ente...	16	14	9	Very dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	No	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	UAH	Ukrainian hryvnia	NaN	NaN	NaN	55.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	A little above average	Yes, because I see value in code review	NaN	Yes, it's part of our process	Not sure	I have little or no influence	C++;HTML/CSS;Java;JavaScript;Python;SQL;VBA	HTML/CSS;Java;JavaScript;SQL;WebAssembly	Couchbase;MongoDB;MySQL;Oracle;PostgreSQL;SQLite	Couchbase;Firebase;MongoDB;MySQL;Oracle;Postgr...	Android;Linux;MacOS;Slack;Windows	Android;Docker;Kubernetes;Linux;Slack	Django;Express;Flask;jQuery;React.js;Spring	Flask;jQuery;React.js;Spring	Cordova;Node.js	Apache Spark;Hadoop;Node.js;React Native	IntelliJ;Notepad++;Vim	Linux-based	Outside of work, for personal projects	Not at all	NaN	Yes	Also Yes	Yes	Facebook	In real life (in person)	Username	I don't remember	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88377	NaN	True	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	Not employed, and not looking for work	Canada	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;JavaScript;Other(s):	C++;HTML/CSS;JavaScript;SQL;WebAssembly;Other(s):	Firebase;SQLite	Firebase;MySQL;SQLite	Linux	Google Cloud Platform;Linux	jQuery	jQuery;Vue.js	Node.js	React Native;Unity 3D;Unreal Engine	Atom;Visual Studio;Visual Studio Code	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	What?	YouTube	NaN	Username	I don't remember	A few times per week	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	I have never participated in Q&A on Stack Over...	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not at all	NaN	Tech articles written by other developers;Tech...	NaN	Man	No	NaN	NaN	No	Appropriate in length	Easy
88601	NaN	False	Never	The quality of OSS and closed source software ...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88802	NaN	False	Never	NaN	Employed full-time	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88816	NaN	False	Never	OSS is, on average, of HIGHER quality than pro...	Independent contractor, freelancer, or self-em...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88863	NaN	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, and not looking for work	Spain	Yes, full-time	Professional degree (JD, MD, etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	NaN	NaN	8	11	3	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;HTML/CSS;Java;JavaScript...	C;C++;C#;HTML/CSS;Java;Objective-C;SQL;Swift	DynamoDB;MariaDB;Microsoft SQL Server;MySQL	MariaDB;MySQL;Oracle;SQLite	iOS;Linux;MacOS;WordPress	Android;AWS;Google Cloud Platform;IBM Cloud or...	Django;jQuery;React.js	Django	Unity 3D;Unreal Engine	NaN	Android Studio;Atom;Eclipse;NetBeans;Notepad++...	Linux-based	Production	NaN	Useful for decentralized currency (i.e., Bitcoin)	Yes	Also Yes	Yes	WhatsApp	In real life (in person)	NaN	2012	Daily or almost daily	Find answers to specific questions;Learn how t...	6-10 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Indu...	18.0	Man	No	Straight / Heterosexual	Hispanic or Latino/Latina;White or of European...	No	Appropriate in length	Easy

88883 rows × 84 columns

Add/Remove Columns¶

In [99]:

ex_df["full_name"]=ex_df["first"]+" "+ex_df["last"]
ex_df

Out[99]:

	email	first	last	full_name
0	corey@gmail.com	corey	schafer	corey schafer
1	john@gmail.com	john	smith	john smith
2	janesmith@gmail.com	jane	smith	jane smith

In [100]:

ex_df.drop(columns=["first","last"],inplace=True)
ex_df

Out[100]:

	email	full_name
0	corey@gmail.com	corey schafer
1	john@gmail.com	john smith
2	janesmith@gmail.com	jane smith

In [101]:

ex_df["full_name"].str.split(" ",expand=True) #when we want to split a column into two

Out[101]:

	0	1
0	corey	schafer
1	john	smith
2	jane	smith

In [102]:

ex_df[["first","last"]] = ex_df["full_name"].str.split(" ",expand=True)

In [103]:

ex_df

Out[103]:

	email	full_name	first	last
0	corey@gmail.com	corey schafer	corey	schafer
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith

In [104]:

#ex_df.append({"first":"Tony"})

In [105]:

ex_df.append({"first":"Tony"},ignore_index=True) #adding rows

Out[105]:

	email	full_name	first	last
0	corey@gmail.com	corey schafer	corey	schafer
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith
3	NaN	NaN	Tony	NaN

In [106]:

people={"first":["Tony","Steve"],
        "last":["Stark","Rogers"],
        "email":["IronMan@avengers.com","Cap@avengers.com"]}
ex_df2 = pd.DataFrame(people)
ex_df2

Out[106]:

	first	last	email
0	Tony	Stark	IronMan@avengers.com
1	Steve	Rogers	Cap@avengers.com

In [107]:

#take note we have conflicting indices

In [108]:

ex_df.append(ex_df2,ignore_index=True)

Out[108]:

	email	full_name	first	last
0	corey@gmail.com	corey schafer	corey	schafer
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

In [109]:

ex_df = ex_df.append(ex_df2,ignore_index=True)

In [110]:

ex_df

Out[110]:

	email	full_name	first	last
0	corey@gmail.com	corey schafer	corey	schafer
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

In [111]:

ex_df.drop(index=4)

Out[111]:

	email	full_name	first	last
0	corey@gmail.com	corey schafer	corey	schafer
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith
3	IronMan@avengers.com	NaN	Tony	Stark

In [112]:

filt = ex_df["last"]=="smith"
ex_df.drop(index=ex_df[filt].index)

Out[112]:

	email	full_name	first	last
0	corey@gmail.com	corey schafer	corey	schafer
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

Sorting Data¶

In [113]:

ex_df.sort_values(by="last") #default is ascending order

Out[113]:

	email	full_name	first	last
4	Cap@avengers.com	NaN	Steve	Rogers
3	IronMan@avengers.com	NaN	Tony	Stark
0	corey@gmail.com	corey schafer	corey	schafer
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith

In [114]:

ex_df.sort_values(by="last",ascending=False)

Out[114]:

	email	full_name	first	last
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith
0	corey@gmail.com	corey schafer	corey	schafer
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

In [115]:

ex_df.sort_values(by=["last","first"],ascending=False) #if two elements of "last" are same then sort by "first"

Out[115]:

	email	full_name	first	last
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith
0	corey@gmail.com	corey schafer	corey	schafer
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

In [116]:

ex_df.sort_values(by=["last","first"],ascending=[False,True]) #sorts "last" in descending and "first" in ascending

Out[116]:

	email	full_name	first	last
2	janesmith@gmail.com	jane smith	jane	smith
1	john@gmail.com	john smith	john	smith
0	corey@gmail.com	corey schafer	corey	schafer
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

In [117]:

ex_df.sort_values(by=["last","first"],ascending=[False,True] , inplace=True)

In [118]:

ex_df

Out[118]:

	email	full_name	first	last
2	janesmith@gmail.com	jane smith	jane	smith
1	john@gmail.com	john smith	john	smith
0	corey@gmail.com	corey schafer	corey	schafer
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

In [119]:

ex_df.sort_index()

Out[119]:

	email	full_name	first	last
0	corey@gmail.com	corey schafer	corey	schafer
1	john@gmail.com	john smith	john	smith
2	janesmith@gmail.com	jane smith	jane	smith
3	IronMan@avengers.com	NaN	Tony	Stark
4	Cap@avengers.com	NaN	Steve	Rogers

In [120]:

ex_df["last"].sort_values()

Out[120]:

4     Rogers
3      Stark
0    schafer
2      smith
1      smith
Name: last, dtype: object

In [121]:

df.sort_values(by=["Country","SalaryUSD"], ascending=[True,False],inplace=True)

In [122]:

df[["Country","SalaryUSD"]].head(50)

Out[122]:

	Country	SalaryUSD
Respondent
63129	Afghanistan	1000000.0
50499	Afghanistan	153216.0
39258	Afghanistan	19152.0
58450	Afghanistan	17556.0
7085	Afghanistan	14364.0
22450	Afghanistan	7980.0
48436	Afghanistan	4464.0
10746	Afghanistan	3996.0
8149	Afghanistan	1596.0
29736	Afghanistan	1116.0
722	Afghanistan	0.0
28638	Afghanistan	0.0
6417	Afghanistan	NaN
7353	Afghanistan	NaN
12310	Afghanistan	NaN
26340	Afghanistan	NaN
29045	Afghanistan	NaN
33178	Afghanistan	NaN
37802	Afghanistan	NaN
40000	Afghanistan	NaN
43106	Afghanistan	NaN
44403	Afghanistan	NaN
45854	Afghanistan	NaN
47110	Afghanistan	NaN
49702	Afghanistan	NaN
50767	Afghanistan	NaN
51859	Afghanistan	NaN
58760	Afghanistan	NaN
59560	Afghanistan	NaN
60569	Afghanistan	NaN
60946	Afghanistan	NaN
62168	Afghanistan	NaN
62525	Afghanistan	NaN
74386	Afghanistan	NaN
80926	Afghanistan	NaN
85715	Afghanistan	NaN
85825	Afghanistan	NaN
86362	Afghanistan	NaN
86932	Afghanistan	NaN
87091	Afghanistan	NaN
88582	Afghanistan	NaN
88731	Afghanistan	NaN
2782	Afghanistan	NaN
63019	Afghanistan	NaN
38308	Albania	187668.0
3787	Albania	114550.0
9270	Albania	74474.0
6716	Albania	60000.0
10303	Albania	57300.0
66280	Albania	41244.0

In [123]:

 df["SalaryUSD"].nlargest(10)

Out[123]:

Respondent
25983    2000000.0
87896    2000000.0
22013    2000000.0
28243    2000000.0
72732    2000000.0
78151    2000000.0
80200    2000000.0
52132    2000000.0
75561    2000000.0
32250    2000000.0
Name: SalaryUSD, dtype: float64

In [124]:

df.nlargest(10,"SalaryUSD")

Out[124]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
25983	I am a developer by profession	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Canada	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Received on-the-job training in software devel...	10,000 or more employees	Developer, full-stack	8	16	2	Very satisfied	Very satisfied	Somewhat confident	No	No	I’m not actively looking, but I am open to new...	3-4 years ago	Solve a brain-teaser style puzzle;Interview wi...	No	Remote work options;Opportunities for professi...	My job status changed (promotion, new job, etc.)	USD	United States dollar	65500.0	Weekly	2000000.0	45.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Meeting...	A few days each month	Office	Average	Yes, because I see value in code review	2.0	Yes, it's part of our process	Developers and management have nearly equal in...	I have little or no influence	Bash/Shell/PowerShell;C++;C#;HTML/CSS;JavaScri...	Bash/Shell/PowerShell;C#;F#;Python;SQL;TypeScript	Microsoft SQL Server;MySQL	Microsoft SQL Server;MySQL	AWS;Docker;Heroku;Linux;MacOS;Raspberry Pi;Win...	AWS;Docker;Linux;MacOS;Windows	Angular/Angular.js	Angular/Angular.js;Flask;React.js	.NET;.NET Core;Node.js	.NET Core;Node.js	Notepad++;Sublime Text;Vim;Visual Studio;Visua...	Windows	Development;Production	NaN	Useful for decentralized currency (i.e., Bitcoin)	No	Yes	Yes	Reddit	In real life (in person)	Username	2013	Daily or almost daily	Find answers to specific questions;Learn how t...	1-2 times per week	Stack Overflow was much faster	11-30 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	NaN	24.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
87896	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	Germany	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	2-9 employees	Developer, full-stack	9	17	8	Very satisfied	Very satisfied	Somewhat confident	No	I am already a manager	I am not interested in new job opportunities	More than 4 years ago	Write any code;Interview with people in senior...	No	Office environment or company culture;Diversit...	Something else changed (education, award, medi...	USD	United States dollar	55000.0	Weekly	2000000.0	40.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Meeting...	All or almost all the time (I'm full-time remote)	Home	A little above average	Yes, because I see value in code review	3.0	No, but I think we should	Developers and management have nearly equal in...	I have a great deal of influence	HTML/CSS;JavaScript;PHP;SQL	HTML/CSS;JavaScript;PHP;Ruby;SQL;TypeScript	MariaDB;MySQL;PostgreSQL	MariaDB;MySQL;PostgreSQL	WordPress	NaN	jQuery;Laravel;Vue.js	Express;Laravel;Ruby on Rails;Vue.js	NaN	Node.js	Sublime Text;Vim;Visual Studio Code	Linux-based	I do not use containers	Not at all	Useful for immutable record keeping outside of...	No	Yes	Yes	Twitter	Online	Handle	2011	Daily or almost daily	Find answers to specific questions	3-5 times per week	Stack Overflow was much faster	0-10 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	32.0	Man	No	Gay or Lesbian	White or of European descent	No	Appropriate in length	Neither easy nor difficult
22013	I am a developer by profession	True	Never	The quality of OSS and closed source software ...	Employed full-time	India	No	Professional degree (JD, MD, etc.)	A natural science (ex. biology, chemistry, phy...	Taken an online course in programming or softw...	2-9 employees	Academic researcher;Data scientist or machine ...	3	26	3	Very satisfied	Very satisfied	NaN	NaN	NaN	I am not interested in new job opportunities	3-4 years ago	Write code by hand (e.g., on a whiteboard);Com...	No	Financial performance or funding status of the...	Something else changed (education, award, medi...	USD	United States dollar	1500000.0	Weekly	2000000.0	50.0	There's no schedule or spec; I work on what se...	Inadequate access to necessary tools;Meetings;...	All or almost all the time (I'm full-time remote)	Home	A little below average	Yes, because I see value in code review	10.0	Yes, it's part of our process	Developers typically have the most influence o...	I have a great deal of influence	Assembly;Bash/Shell/PowerShell;C;C++;Java;Java...	Python;Swift	Cassandra;Microsoft SQL Server;Oracle	Cassandra	Android;Arduino;Google Cloud Platform;IBM Clou...	IBM Cloud or Watson;Windows	jQuery	NaN	TensorFlow	TensorFlow	Visual Studio;Visual Studio Code	Windows	Development	Implementing our own cryptocurrency	Useful across many domains and could change ma...	Yes	Yes	No	YouTube	Online	Screen Name	NaN	A few times per week	Find answers to specific questions	1-2 times per week	Stack Overflow was much faster	60+ minutes	Yes	I have never participated in Q&A on Stack Over...	Yes	Yes	Yes, definitely	A lot more welcome now than last year	Tech articles written by other developers;Indu...	NaN	Man	No	Straight / Heterosexual	NaN	Yes	Too long	Easy
28243	I am a developer by profession	True	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Independent contractor, freelancer, or self-em...	India	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	2-9 employees	Developer, back-end;Developer, full-stack;Deve...	25	9	14	Neither satisfied nor dissatisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles;Interview ...	Yes	Financial performance or funding status of the...	I heard about a job opportunity (from a recrui...	USD	United States dollar	2000000.0	Yearly	2000000.0	20.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Not eno...	All or almost all the time (I'm full-time remote)	Office	Far above average	Yes, because I see value in code review	NaN	NaN	NaN	NaN	C++;C#;Java;JavaScript;Objective-C;Python	C;C++;C#	PostgreSQL;SQLite	SQLite	Android;AWS;iOS;MacOS;Windows	Android;Linux	NaN	NaN	Node.js;Pandas;Unity 3D	Unity 3D;Unreal Engine	Android Studio;Coda;PyCharm;Visual Studio;Visu...	MacOS	I do not use containers	NaN	An irresponsible use of resources	No	Yes	No	I don't use social media	In real life (in person)	Username	2008	Daily or almost daily	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	60+ minutes	Yes	Multiple times per day	Yes	No, and I don't know what those are	Yes, definitely	A lot less welcome now than last year	Tech meetups or events in your area	NaN	NaN	NaN	Straight / Heterosexual	NaN	Yes	Too short	Easy
72732	I am not primarily a developer, but I write co...	False	Less than once a month but more than once per ...	OSS is, on average, of LOWER quality than prop...	NaN	India	Yes, full-time	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Contributed to open source software	NaN	NaN	4	20	5	Slightly satisfied	Slightly satisfied	NaN	NaN	NaN	I am actively looking for a job	More than 4 years ago	Write any code	No	Remote work options;Opportunities for professi...	I was preparing for a job search	USD	United States dollar	80000.0	Weekly	2000000.0	40.0	There is a schedule and/or spec (made by me or...	Lack of support from management;Meetings;Time ...	Less than once per month / Never	Office	A little above average	Yes, because I see value in code review	10.0	Yes, it's part of our process	Not sure	I have little or no influence	Java;Python;SQL	NaN	MongoDB;Microsoft SQL Server;MySQL;Oracle;Post...	MySQL	Android;Windows	AWS;Docker	NaN	Angular/Angular.js;React.js	NaN	NaN	Eclipse;Notepad++;PyCharm	Windows	Testing	NaN	NaN	Yes	Yes	No	WhatsApp	Neither	NaN	2017	A few times per month or weekly	Get a sense of belonging to the developer comm...	1-2 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	A few times per month or weekly	Yes	No, and I don't know what those are	Not sure	A lot less welcome now than last year	Tech articles written by other developers;Tech...	NaN	Man	No	NaN	NaN	Yes	Too long	Easy
78151	I am a developer by profession	True	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Mexico	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	15	18	10	Very satisfied	Very satisfied	Somewhat confident	No	Yes	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	Yes	Languages, frameworks, and other technologies ...	My job status changed (promotion, new job, etc.)	USD	United States dollar	95000.0	Weekly	2000000.0	30.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Distrac...	Less than once per month / Never	Office	A little above average	No	NaN	No, but I think we should	The CTO, CIO, or other management purchase new...	I have little or no influence	C#;HTML/CSS;JavaScript;SQL;TypeScript	C#;TypeScript;WebAssembly	Microsoft SQL Server	Microsoft SQL Server	Windows	Android;Arduino;Raspberry Pi	Angular/Angular.js;ASP.NET;jQuery;Vue.js	Angular/Angular.js;Vue.js	.NET	.NET Core;Xamarin	Visual Studio	Windows	I do not use containers	Non-currency applications of blockchain	Useful for immutable record keeping outside of...	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2010	Multiple times per day	Learn how to do things I didn’t necessarily lo...	1-2 times per week	Stack Overflow was much faster	60+ minutes	Yes	A few times per month or weekly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	32.0	Man	No	Straight / Heterosexual	Hispanic or Latino/Latina	No	Appropriate in length	Easy
80200	I am a developer by profession	True	Never	OSS is, on average, of LOWER quality than prop...	Employed full-time	Netherlands	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Received on-the-job training in software devel...	1,000 to 4,999 employees	Developer, front-end;Developer, mobile	8	17	4	Very satisfied	Very satisfied	Somewhat confident	Yes	Yes	I am not interested in new job opportunities	More than 4 years ago	Complete a take-home project;Interview with pe...	No	Office environment or company culture;Diversit...	My job status changed (promotion, new job, etc.)	USD	United States dollar	120000.0	Weekly	2000000.0	45.0	There is a schedule and/or spec (made by me or...	Distracting work environment	A few days each month	Office	Average	Yes, because I see value in code review	15.0	Yes, it's not part of our process but the deve...	Not sure	I have little or no influence	Objective-C;Swift	Swift	NaN	NaN	Arduino;iOS;MacOS	Arduino;iOS;MacOS;Slack	NaN	NaN	NaN	NaN	Sublime Text;Vim;Xcode	MacOS	I do not use containers	Not at all	Useful for immutable record keeping outside of...	Yes	SIGH	Yes	Instagram	In real life (in person)	Handle	2011	Multiple times per day	Find answers to specific questions	1-2 times per week	Stack Overflow was slightly faster	NaN	No	NaN	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Neutral	NaN	Tech articles written by other developers	25.0	Woman	No	Bisexual	White or of European descent	No	Appropriate in length	Easy
52132	I am a developer by profession	True	Less than once a month but more than once per ...	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Peru	No	Some college/university study without earning ...	I never declared a major	Completed an industry certification program (e...	100 to 499 employees	Database administrator;Developer, back-end;Dev...	26	15	24	Slightly satisfied	Slightly dissatisfied	Not at all confident	No	I am already a manager	I am not interested in new job opportunities	3-4 years ago	Write any code;Interview with people in peer r...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	75000.0	Weekly	2000000.0	55.0	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings	All or almost all the time (I'm full-time remote)	Home	Far above average	Yes, because I see value in code review	6.0	Yes, it's not part of our process but the deve...	Developers typically have the most influence o...	I have a great deal of influence	C++;HTML/CSS;JavaScript;Python;SQL;TypeScript	C++;HTML/CSS;JavaScript;Rust;SQL;TypeScript;We...	Firebase;PostgreSQL	Firebase;PostgreSQL	Android;AWS;Docker;Linux;Microsoft Azure;Slack...	Android;Arduino;Kubernetes;Linux;Raspberry Pi	Express	Express	Ansible;Cordova;Node.js	Ansible;Cordova;Node.js	Vim;Visual Studio Code	Linux-based	Development;Testing;Production	Not at all	A passing fad	No	SIGH	What?	WhatsApp	Neither	Username	2008	A few times per week	Find answers to specific questions	1-2 times per week	Stack Overflow was much faster	11-30 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	48.0	Man	NaN	NaN	Black or of African descent;East Asian;Hispani...	Yes	Appropriate in length	Easy
75561	I am a developer by profession	True	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	Employed full-time	Singapore	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	A humanities discipline (ex. literature, histo...	Taken an online course in programming or softw...	10,000 or more employees	Developer, back-end;Developer, front-end;Devel...	19	10	16	Very satisfied	Very satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	1-2 years ago	Complete a take-home project;Interview with pe...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	223000.0	Monthly	2000000.0	38.0	There's no schedule or spec; I work on what se...	Meetings;Non-work commitments (parenting, scho...	More than half, but not all, the time	Home	Far above average	No	NaN	Yes, it's part of our process	Not sure	I have little or no influence	Bash/Shell/PowerShell;HTML/CSS;JavaScript;Pyth...	Bash/Shell/PowerShell;F#;HTML/CSS;JavaScript;P...	DynamoDB;Elasticsearch;PostgreSQL	DynamoDB;Elasticsearch;PostgreSQL	AWS;Linux;MacOS	AWS;Linux;MacOS	React.js	React.js	Node.js	Node.js;React Native	Vim;Visual Studio Code	MacOS	Development	Non-currency applications of blockchain	NaN	No	Yes	Yes	Twitter	In real life (in person)	Username	2008	Daily or almost daily	Find answers to specific questions;Contribute ...	3-5 times per week	The other resource was much faster	11-30 minutes	Yes	A few times per week	Yes	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area	37.0	Man	No	Straight / Heterosexual	White or of European descent	Yes	Appropriate in length	Easy
32250	I am a developer by profession	True	Once a month or more often	The quality of OSS and closed source software ...	Employed full-time	Switzerland	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken a part-time in-person course in programm...	2-9 employees	Academic researcher;Data scientist or machine ...	16	14	9	Slightly satisfied	Very satisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	1-2 years ago	Write any code;Write code by hand (e.g., on a ...	Yes	Office environment or company culture;Opportun...	Something else changed (education, award, medi...	USD	United States dollar	200000.0	Monthly	2000000.0	50.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Non-wor...	All or almost all the time (I'm full-time remote)	Home	Far above average	Yes, because I see value in code review	20.0	Yes, it's part of our process	Developers typically have the most influence o...	I have a great deal of influence	C#;Go;HTML/CSS;JavaScript;Python;SQL;TypeScript	C#;HTML/CSS;JavaScript	DynamoDB;Elasticsearch;Redis	DynamoDB;Elasticsearch	Android;AWS;Docker;Google Cloud Platform;IBM C...	AWS;Linux	ASP.NET;Express;React.js;Vue.js	Vue.js	.NET Core;Node.js;TensorFlow	Node.js	Visual Studio Code	Linux-based	Development;Testing;Production;Outside of work...	NaN	Useful across many domains and could change ma...	Yes	Also Yes	Yes	Twitter	Online	NaN	2008	A few times per week	Find answers to specific questions;Learn how t...	Less than once per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per week	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Industry news about technologies you're intere...	30.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy

In [125]:

df.nsmallest(10,"SalaryUSD")

Out[125]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
722	I am a developer by profession	False	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	NaN	Afghanistan	NaN	NaN	NaN	NaN	NaN	NaN	Less than 1 year	Younger than 5 years	Less than 1 year	Very satisfied	Very satisfied	NaN	NaN	NaN	I am not interested in new job opportunities	NA - I am an independent contractor or self em...	NaN	No	NaN	I heard about a job opportunity (from a recrui...	AFN	Afghan afghani	1.0	Yearly	0.0	1.0	NaN	NaN	Less than once per month / Never	Other place, such as a coworking space or cafe	Far above average	Yes, because I see value in code review	1.0	No, and I'm glad we don't	NaN	I have little or no influence	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	No	SIGH	What?	NaN	Neither	NaN	2008	Multiple times per day	NaN	More than 10 times per week	The other resource was much faster	60+ minutes	Not sure / can't remember	NaN	Yes	Yes	Not sure	Not applicable - I did not use Stack Overflow ...	NaN	1.0	NaN	NaN	NaN	NaN	NaN	Too long	Difficult
28638	I am a developer by profession	True	Less than once a month but more than once per ...	OSS is, on average, of HIGHER quality than pro...	Independent contractor, freelancer, or self-em...	Afghanistan	NaN	Other doctoral degree (Ph.D, Ed.D., etc.)	I never declared a major	NaN	NaN	NaN	36	13	16	Very dissatisfied	Very dissatisfied	NaN	NaN	NaN	I am not interested in new job opportunities	NaN	NaN	NaN	NaN	NaN	AFN	Afghan afghani	3.0	Monthly	0.0	2.0	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C++;Java;Objective-C;O...	Bash/Shell/PowerShell;C;C++;Java;Objective-C;O...	MySQL;SQLite	MySQL;SQLite	Android;iOS;Linux;MacOS	Android;iOS;Linux	jQuery	NaN	TensorFlow;Unity 3D	Hadoop;TensorFlow;Unity 3D	Emacs	Linux-based	I do not use containers	NaN	Useful for immutable record keeping outside of...	No	Yes	No	NaN	In real life (in person)	Username	2008	Daily or almost daily	Find answers to specific questions	6-10 times per week	The other resource was much faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	No, not at all	A lot less welcome now than last year	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Too long	NaN
13825	I am a developer by profession	True	Never	The quality of OSS and closed source software ...	NaN	Algeria	NaN	NaN	NaN	NaN	NaN	NaN	13	83	Less than 1 year	Very dissatisfied	Very satisfied	NaN	NaN	NaN	I am actively looking for a job	NA - I am an independent contractor or self em...	Complete a take-home project;Solve a brain-tea...	Yes	Financial performance or funding status of the...	Something else changed (education, award, medi...	AFN	Afghan afghani	1.0	Yearly	0.0	168.0	NaN	Being tasked with non-development work;Non-wor...	About half the time	Home	Far above average	No	NaN	Yes, it's part of our process	NaN	I have a great deal of influence	Other(s):	Other(s):	Other(s):	Other(s):	Other(s):	Other(s):	Other(s):	Other(s):	Other(s):	Other(s):	NaN	NaN	I do not use containers	NaN	An irresponsible use of resources	No	Yes	Yes	NaN	Neither	NaN	2015	A few times per month or weekly	Find answers to specific questions	1-2 times per week	They were about the same	NaN	Yes	I have never participated in Q&A on Stack Over...	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	No, not at all	A lot less welcome now than last year	NaN	56.0	NaN	NaN	NaN	NaN	Yes	Too short	Easy
40913	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	Armenia	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	2-9 employees	Developer, desktop or enterprise applications;...	7	15	4	Slightly satisfied	Slightly satisfied	Very confident	Not sure	Not sure	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	Something else changed (education, award, medi...	AMD	Armenian dram	0.0	Monthly	0.0	40.0	There's no schedule or spec; I work on what se...	Meetings;Non-work commitments (parenting, scho...	Less than half the time, but at least one day ...	Office	A little above average	Yes, because I see value in code review	3.0	No, and I'm glad we don't	Developers and management have nearly equal in...	I have a great deal of influence	Bash/Shell/PowerShell;C;C++;Java;JavaScript;Py...	C;C++;Java;Python;SQL;TypeScript	Elasticsearch;MariaDB;MongoDB;MySQL;Redis	MariaDB;MongoDB;MySQL	Arduino;AWS;Docker;Linux;MacOS;Raspberry Pi;Sl...	Arduino;Linux;MacOS;Raspberry Pi	ASP.NET;Django;Flask	Django;Flask	.NET;Node.js;TensorFlow;Torch/PyTorch	Node.js	IntelliJ;IPython / Jupyter;Sublime Text;Vim	Linux-based	Production	Not at all	A passing fad	No	SIGH	Yes	Facebook	In real life (in person)	Username	2013	Multiple times per day	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	0-10 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers	22.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Easy
30630	I am a developer by profession	True	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	Employed full-time	Australia	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	NaN	Taken an online course in programming or softw...	2-9 employees	Database administrator;Developer, back-end;Dev...	35	7	30	Neither satisfied nor dissatisfied	Very satisfied	NaN	NaN	NaN	I am not interested in new job opportunities	NA - I am an independent contractor or self em...	Interview with people in peer roles;Interview ...	No	Industry that I'd be working in;Office environ...	I heard about a job opportunity (from a recrui...	AUD	Australian dollar	0.0	Yearly	0.0	80.0	There's no schedule or spec; I work on what se...	Distracting work environment;Non-work commitme...	It's complicated	Other place, such as a coworking space or cafe	A little above average	Yes, because I see value in code review	10.0	No, but I think we should	NaN	I have a great deal of influence	Bash/Shell/PowerShell;HTML/CSS;JavaScript;Ruby	Elixir;Ruby;TypeScript	PostgreSQL	PostgreSQL	AWS;iOS	Android;AWS;iOS	Ruby on Rails;Vue.js	Ruby on Rails;Vue.js	Chef;React Native	React Native	Sublime Text;TextMate;Vim;Xcode	MacOS	I do not use containers	Not at all	An irresponsible use of resources	Yes	Fortunately, someone else has that title	No	Reddit	Online	Username	I don't remember	A few times per month or weekly	Find answers to specific questions	1-2 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	No, not really	A lot less welcome now than last year	NaN	43.0	NaN	No	NaN	NaN	No	Appropriate in length	Easy
69049	I am a developer by profession	False	Never	The quality of OSS and closed source software ...	Independent contractor, freelancer, or self-em...	Australia	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Received on-the-job training in software devel...	2-9 employees	Developer, full-stack;Engineering manager	17	12	14	Slightly satisfied	Slightly satisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	1-2 years ago	Write any code;Write code by hand (e.g., on a ...	No	Specific department or team I'd be working on;...	My job status changed (promotion, new job, etc.)	AUD	Australian dollar	0.0	Yearly	0.0	25.0	There is a schedule and/or spec (made by me or...	Lack of support from management;Time spent com...	All or almost all the time (I'm full-time remote)	Home	A little above average	Yes, because I see value in code review	1.0	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C#;Go;HTML/CSS;JavaScr...	Dart;Elixir;JavaScript;Kotlin;Python;R;Ruby;Rust	PostgreSQL;SQLite	Cassandra;PostgreSQL;Redis	Android;Arduino;Google Cloud Platform;iOS;Linu...	Google Cloud Platform;Heroku;Kubernetes;Linux;...	Ruby on Rails	Ruby on Rails;Vue.js	.NET;React Native	Ansible;Flutter;Hadoop;Pandas;React Native	IntelliJ;Visual Studio;Visual Studio Code;Xcode	MacOS	Production	NaN	A passing fad	No	Also Yes	Yes	Reddit	In real life (in person)	Username	2008	A few times per week	Find answers to specific questions	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Not sure / can't remember	NaN	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not at all	Just as welcome now as I felt last year	NaN	34.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
19340	I am not primarily a developer, but I write co...	True	Less than once a month but more than once per ...	OSS is, on average, of HIGHER quality than pro...	NaN	Austria	Yes, full-time	Bachelor’s degree (BA, BS, B.Eng., etc.)	Fine arts or performing arts (ex. graphic desi...	Taken an online course in programming or softw...	NaN	NaN	10	20	5	Neither satisfied nor dissatisfied	Slightly satisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	3-4 years ago	Interview with people in peer roles;Interview ...	No	Office environment or company culture;Opportun...	I was preparing for a job search	CHF	Swiss franc	0.0	Monthly	0.0	35.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	Average	Yes, because I see value in code review	5.0	Yes, it's part of our process	Developers and management have nearly equal in...	I have a great deal of influence	Assembly;C;Elixir;HTML/CSS;Java;JavaScript;Pyt...	Clojure	PostgreSQL;SQLite	NaN	Arduino;Raspberry Pi	Arduino;Raspberry Pi	Other(s):	Other(s):	NaN	NaN	IntelliJ;Sublime Text	Linux-based	Development;Testing;Production;Outside of work...	NaN	Useful for immutable record keeping outside of...	No	Also Yes	Yes	NaN	In real life (in person)	Username	2010	A few times per week	Find answers to specific questions;Learn how t...	1-2 times per week	They were about the same	NaN	Yes	I have never participated in Q&A on Stack Over...	Yes	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	NaN	32.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
16214	I am not primarily a developer, but I write co...	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Bangladesh	Yes, part-time	Master’s degree (MA, MS, M.Eng., MBA, etc.)	A natural science (ex. biology, chemistry, phy...	NaN	20 to 99 employees	Engineer, data;Engineering manager;Product man...	20	16	20	Slightly satisfied	Slightly satisfied	NaN	NaN	NaN	I am not interested in new job opportunities	3-4 years ago	Interview with people in peer roles	No	Financial performance or funding status of the...	I was preparing for a job search	EUR	European Euro	0.0	Yearly	0.0	10.0	There's no schedule or spec; I work on what se...	Lack of support from management;Meetings;Time ...	Less than once per month / Never	Other place, such as a coworking space or cafe	Far above average	No	NaN	No, and I'm glad we don't	Developers and management have nearly equal in...	I have a great deal of influence	Java;PHP;Other(s):	Java;PHP;Other(s):	MariaDB;MySQL	MariaDB;MySQL	Android;Linux;Raspberry Pi;Windows;WordPress	Android;Linux;Raspberry Pi;Windows;WordPress	jQuery;Spring	jQuery;Spring	NaN	NaN	Android Studio;Eclipse;IntelliJ;NetBeans	Linux-based	I do not use containers	Not at all	NaN	No	Yes	What?	WhatsApp	Neither	NaN	2008	Daily or almost daily	Find answers to specific questions;Contribute ...	3-5 times per week	They were about the same	NaN	Yes	A few times per month or weekly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Cour...	99.0	NaN	No	NaN	NaN	NaN	Too long	Easy
29355	I am a developer by profession	True	Never	OSS is, on average, of LOWER quality than prop...	Independent contractor, freelancer, or self-em...	Bosnia and Herzegovina	Yes, part-time	Some college/university study without earning ...	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	Just me - I am a freelancer, sole proprietor, ...	NaN	10	10	6	Neither satisfied nor dissatisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I am not interested in new job opportunities	Less than a year ago	Interview with people in senior / management r...	No	Industry that I'd be working in;Specific depar...	Something else changed (education, award, medi...	BAM	Bosnia and Herzegovina convertible mark	0.0	Yearly	0.0	80.0	There's no schedule or spec; I work on what se...	Being tasked with non-development work;Inadequ...	It's complicated	Other place, such as a coworking space or cafe	Far above average	Yes, because I see value in code review	20.0	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C++;Python;Ruby;Other(s):	Assembly;Bash/Shell/PowerShell;C;C++;C#;Go;Oth...	NaN	NaN	Linux;MacOS;Microsoft Azure;Raspberry Pi;Windows	Android;Arduino;AWS;iOS;Linux;MacOS;Raspberry ...	NaN	NaN	Pandas;TensorFlow;Torch/PyTorch;Unity 3D;Unrea...	.NET;Torch/PyTorch;Unity 3D	Android Studio;Emacs;PyCharm;Visual Studio;Vis...	Linux-based	I do not use containers	NaN	A passing fad	Yes	Also Yes	Yes	YouTube	In real life (in person)	Username	2013	A few times per month or weekly	Find answers to specific questions;Learn how t...	Less than once per week	They were about the same	NaN	Not sure / can't remember	NaN	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	NaN	No	NaN	NaN	NaN	Too long	Easy
1685	I am a developer by profession	True	Less than once a month but more than once per ...	The quality of OSS and closed source software ...	Independent contractor, freelancer, or self-em...	Brazil	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	Just me - I am a freelancer, sole proprietor, ...	Designer;Developer, full-stack;Developer, game...	8	16	3	Neither satisfied nor dissatisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	Less than a year ago	Interview with people in peer roles;Interview ...	No	Specific department or team I'd be working on;...	I was preparing for a job search	BRL	Brazilian real	0.0	Monthly	0.0	40.0	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Toxic wo...	All or almost all the time (I'm full-time remote)	Home	Average	Yes, because I see value in code review	6.0	NaN	NaN	NaN	Elixir;HTML/CSS;JavaScript;PHP;Python	C#;Kotlin;Python	Cassandra;MariaDB;MongoDB;PostgreSQL	Cassandra;MariaDB;MongoDB;PostgreSQL	Docker;Kubernetes;MacOS	Android;Linux;Windows	Laravel	Django;Flask;Laravel;Vue.js	Node.js	.NET;Node.js;Unity 3D;Other(s):	Android Studio;PHPStorm;PyCharm;Visual Studio ...	Windows	Development	NaN	Useful across many domains and could change ma...	No	Yes	Yes	WhatsApp	In real life (in person)	NaN	2010	A few times per week	Find answers to specific questions	1-2 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	29.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Neither easy nor difficult

Grouping and Aggregating - Analyzing Data¶

In [126]:

df["SalaryUSD"].median()

Out[126]:

57287.0

In [127]:

df.median()

Out[127]:

Hobbyist           1.0
CompTotal      62000.0
SalaryUSD      57287.0
WorkWeekHrs       40.0
CodeRevHrs         4.0
Age               29.0
dtype: float64

In [128]:

df.describe()

Out[128]:

	CompTotal	SalaryUSD	WorkWeekHrs	CodeRevHrs	Age
count	5.594500e+04	5.582300e+04	64503.000000	49790.000000	79210.000000
mean	5.519014e+11	1.271107e+05	42.127197	5.084308	30.336699
std	7.331926e+13	2.841523e+05	37.287610	5.513931	9.178390
min	0.000000e+00	0.000000e+00	1.000000	0.000000	1.000000
25%	2.000000e+04	2.577750e+04	40.000000	2.000000	24.000000
50%	6.200000e+04	5.728700e+04	40.000000	4.000000	29.000000
75%	1.200000e+05	1.000000e+05	44.750000	6.000000	35.000000
max	1.000000e+16	2.000000e+06	4850.000000	99.000000	99.000000

In [129]:

df

Out[129]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
63129	I am a developer by profession	True	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Afghanistan	Yes, full-time	I never completed any formal education	NaN	Taken an online course in programming or softw...	Just me - I am a freelancer, sole proprietor, ...	NaN	Less than 1 year	Younger than 5 years	1	Very dissatisfied	Very dissatisfied	Very confident	Yes	Yes	I am actively looking for a job	I've never had a job	Write any code	Yes	How widely used or impactful my work output wo...	I had a negative experience or interaction at ...	AED	United Arab Emirates dirham	648838511.0	Monthly	1000000.0	168.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work	It's complicated	Other place, such as a coworking space or cafe	Far below average	No	NaN	NaN	NaN	NaN	Assembly;Bash/Shell/PowerShell;Other(s):	Bash/Shell/PowerShell;TypeScript	MySQL	MySQL	Android;iOS;Windows;WordPress;Other(s):	Android;iOS;Windows;WordPress;Other(s):	Angular/Angular.js;Express;jQuery;Laravel;Othe...	Angular/Angular.js;Express;jQuery;Laravel;Othe...	Ansible;Apache Spark;Torch/PyTorch	Apache Spark;Torch/PyTorch	Android Studio;Atom;Coda;Eclipse;Emacs;IPython...	Windows	Outside of work, for personal projects	Accepting Bitcoin or other coins and tokens as...	Useful across many domains and could change ma...	Yes	Yes	Yes	I don't use social media	Online	UserID	NaN	I have never visited Stack Overflow (before to...	Find answers to specific questions	Less than once per week	Stack Overflow was much faster	0-10 minutes	NaN	NaN	No, I didn't know that Stack Overflow had a jo...	NaN	No, not at all	NaN	Tech articles written by other developers	NaN	NaN	NaN	NaN	NaN	Yes	Too short	Easy
50499	I am not primarily a developer, but I write co...	True	Less than once per year	OSS is, on average, of LOWER quality than prop...	Employed full-time	Afghanistan	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	A health science (ex. nursing, pharmacy, radio...	Participated in a full-time developer training...	10,000 or more employees	Data or business analyst;Database administrato...	7	27	7	Very dissatisfied	Very dissatisfied	Somewhat confident	Yes	Yes	I’m not actively looking, but I am open to new...	More than 4 years ago	Write code by hand (e.g., on a whiteboard);Int...	No	Opportunities for professional development;Div...	Something else changed (education, award, medi...	AFN	Afghan afghani	960000.0	Monthly	153216.0	40.0	There is a schedule and/or spec (made by me or...	Lack of support from management;Time spent com...	Less than once per month / Never	Office	A little above average	Yes, because I see value in code review	8.0	Yes, it's not part of our process but the deve...	Developers typically have the most influence o...	I have a great deal of influence	C#;HTML/CSS;JavaScript;SQL;VBA	C#;HTML/CSS;JavaScript;R;SQL;TypeScript;VBA	Microsoft SQL Server;PostgreSQL;SQLite	MariaDB;MongoDB;Microsoft SQL Server;PostgreSQ...	NaN	Android	ASP.NET;jQuery	Angular/Angular.js;ASP.NET;jQuery;React.js;Vue.js	.NET	.NET;.NET Core;Node.js;Xamarin	Visual Studio	Windows	I do not use containers	Implementing our own cryptocurrency	NaN	Yes	Yes	Yes	WhatsApp	Online	Login	2013	A few times per month or weekly	Find answers to specific questions	Less than once per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	34.0	NaN	No	Straight / Heterosexual	NaN	Yes	Too long	Neither easy nor difficult
39258	I am a developer by profession	True	Less than once per year	OSS is, on average, of LOWER quality than prop...	Employed full-time	Afghanistan	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Database administrator;Designer;Developer, bac...	2	22	1	Very satisfied	Very satisfied	Very confident	No	Yes	I’m not actively looking, but I am open to new...	1-2 years ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	Something else changed (education, award, medi...	AFN	Afghan afghani	120000.0	Monthly	19152.0	49.0	There is a schedule and/or spec (made by me or...	Not enough people for the workload;Time spent ...	Less than once per month / Never	Office	Far above average	Yes, because I see value in code review	10.0	Yes, it's not part of our process but the deve...	Developers typically have the most influence o...	I have some influence	HTML/CSS;JavaScript;PHP;SQL;TypeScript	C#;Dart;JavaScript;SQL;TypeScript	MariaDB;MongoDB;MySQL;PostgreSQL	Couchbase;Firebase;MariaDB;MongoDB;Redis	Android;AWS;Linux;Windows;WordPress	Android;AWS;Docker;Heroku;iOS;Linux;Raspberry ...	Angular/Angular.js;Drupal;Express;Laravel	Angular/Angular.js;Drupal;Express	Cordova;Node.js	Hadoop;Node.js	Notepad++;PHPStorm;Vim;Visual Studio Code	Linux-based	Production	NaN	NaN	Yes	Yes	No	Facebook	Online	Login	2016	A few times per week	Find answers to specific questions;Learn how t...	1-2 times per week	Stack Overflow was much faster	0-10 minutes	Yes	Less than once per month or monthly	Yes	NaN	Yes, somewhat	NaN	Tech articles written by other developers;Indu...	26.0	Man	No	Straight / Heterosexual	East Asian;South Asian	Yes	Too long	Easy
58450	I am a developer by profession	True	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Afghanistan	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Developer, back-end;Developer, desktop or ente...	6	20	2	Neither satisfied nor dissatisfied	Slightly satisfied	Very confident	Yes	Yes	I am actively looking for a job	1-2 years ago	Write code by hand (e.g., on a whiteboard);Com...	No	Languages, frameworks, and other technologies ...	I had a negative experience or interaction at ...	AFN	Afghan afghani	110000.0	Monthly	17556.0	40.0	There is a schedule and/or spec (made by me or...	Distracting work environment	It's complicated	Office	Average	Yes, because I see value in code review	8.0	Yes, it's part of our process	Developers typically have the most influence o...	I have little or no influence	Assembly;C;C++;HTML/CSS;Java;PHP;SQL	C;C++;HTML/CSS;Java;JavaScript;SQL;TypeScript	MySQL;PostgreSQL	MySQL;PostgreSQL	Linux;Windows	Windows	NaN	Spring	NaN	Node.js	IntelliJ;Notepad++	Windows	Development	Accepting Bitcoin or other coins and tokens as...	Useful for immutable record keeping outside of...	Yes	Yes	Yes	Facebook	Online	Handle	NaN	Daily or almost daily	Find answers to specific questions;Learn how t...	6-10 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Yes, definitely	A lot less welcome now than last year	Tech articles written by other developers	26.0	Man	No	Straight / Heterosexual	South Asian	No	Too long	Neither easy nor difficult
7085	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Independent contractor, freelancer, or self-em...	Afghanistan	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	2-9 employees	Developer, full-stack	7	19	6	Slightly dissatisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	Less than a year ago	Interview with people in peer roles;Interview ...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	AFN	Afghan afghani	90000.0	Monthly	14364.0	30.0	There's no schedule or spec; I work on what se...	Inadequate access to necessary tools;Lack of s...	About half the time	Office	Average	Yes, because I see value in code review	3.0	NaN	NaN	NaN	HTML/CSS;JavaScript;PHP	JavaScript;Kotlin;PHP	MySQL	MongoDB;MySQL;Redis	Linux;MacOS;Other(s):	Linux;MacOS;Other(s):	jQuery;Laravel;Vue.js	ASP.NET;jQuery;Laravel;Vue.js	NaN	React Native	Sublime Text;Visual Studio Code	MacOS	Development;Outside of work, for personal proj...	NaN	NaN	Yes	Also Yes	Yes	Instagram	Online	UserID	2016	Multiple times per day	Find answers to specific questions;Learn how t...	6-10 times per week	Stack Overflow was much faster	60+ minutes	Yes	A few times per week	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Industry news about technologies you're intere...	26.0	Man	NaN	Straight / Heterosexual	East Asian	Yes	Appropriate in length	Neither easy nor difficult
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88062	NaN	False	Never	OSS is, on average, of LOWER quality than prop...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88076	NaN	False	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88601	NaN	False	Never	The quality of OSS and closed source software ...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88802	NaN	False	Never	NaN	Employed full-time	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88816	NaN	False	Never	OSS is, on average, of HIGHER quality than pro...	Independent contractor, freelancer, or self-em...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

88883 rows × 84 columns

In [130]:

df.sort_index(inplace=True)

In [131]:

df

Out[131]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
1	I am a student who is learning to code	True	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	4	10	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript;Python	C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL	SQLite	MySQL	MacOS;Windows	Android;Arduino;Windows	Django;Flask	Flask;jQuery	Node.js	Node.js	IntelliJ;Notepad++;PyCharm	Windows	I do not use containers	NaN	NaN	Yes	Fortunately, someone else has that title	Yes	Twitter	Online	Username	2017	A few times per month or weekly	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	31-60 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	I am a student who is learning to code	False	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Developer, desktop or enterprise applications;...	NaN	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Financial performance or funding status of the...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C++;HTML/CSS;Python	C++;HTML/CSS;JavaScript;SQL	NaN	MySQL	Windows	Windows	Django	Django	NaN	NaN	Atom;PyCharm	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	Yes	Instagram	Online	Username	2017	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
3	I am not primarily a developer, but I write co...	True	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	Taught yourself a new language, framework, or ...	100 to 499 employees	Designer;Developer, back-end;Developer, front-...	3	22	1	Slightly satisfied	Slightly satisfied	Not at all confident	Not sure	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	THB	Thai baht	23000.0	Monthly	8820.0	40.00	There's no schedule or spec; I work on what se...	Distracting work environment;Inadequate access...	Less than once per month / Never	Home	Average	No	NaN	No, but I think we should	Not sure	I have little or no influence	HTML/CSS	Elixir;HTML/CSS	PostgreSQL	PostgreSQL	NaN	NaN	NaN	Other(s):	NaN	NaN	Vim;Visual Studio Code	Linux-based	I do not use containers	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
4	I am a developer by profession	False	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.00	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
5	I am a developer by profession	True	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Academic researcher;Developer, desktop or ente...	16	14	9	Very dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	No	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	UAH	Ukrainian hryvnia	NaN	NaN	NaN	55.00	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	A little above average	Yes, because I see value in code review	NaN	Yes, it's part of our process	Not sure	I have little or no influence	C++;HTML/CSS;Java;JavaScript;Python;SQL;VBA	HTML/CSS;Java;JavaScript;SQL;WebAssembly	Couchbase;MongoDB;MySQL;Oracle;PostgreSQL;SQLite	Couchbase;Firebase;MongoDB;MySQL;Oracle;Postgr...	Android;Linux;MacOS;Slack;Windows	Android;Docker;Kubernetes;Linux;Slack	Django;Express;Flask;jQuery;React.js;Spring	Flask;jQuery;React.js;Spring	Cordova;Node.js	Apache Spark;Hadoop;Node.js;React Native	IntelliJ;Notepad++;Vim	Linux-based	Outside of work, for personal projects	Not at all	NaN	Yes	Also Yes	Yes	Facebook	In real life (in person)	Username	I don't remember	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88879	I am a developer by profession	True	Never	The quality of OSS and closed source software ...	Employed full-time	Finland	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	20 to 99 employees	Developer, desktop or enterprise applications;...	17	16	7	Slightly satisfied	Neither satisfied nor dissatisfied	Not at all confident	Not sure	I am already a manager	I’m not actively looking, but I am open to new...	More than 4 years ago	Complete a take-home project;Interview with pe...	No	Languages, frameworks, and other technologies ...	I had a negative experience or interaction at ...	EUR	European Euro	6000.0	Monthly	82488.0	37.75	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Distrac...	Less than once per month / Never	Home	Far above average	Yes, because I see value in code review	10.0	Yes, it's part of our process	Developers and management have nearly equal in...	I have little or no influence	Bash/Shell/PowerShell;C++;Python	C++	NaN	NaN	Android;Linux;Windows	Linux;Windows	NaN	NaN	NaN	TensorFlow;Unity 3D;Unreal Engine	Android Studio;Notepad++;Vim;Visual Studio	Windows	I do not use containers	Not at all	A passing fad	Yes	NaN	NaN	YouTube	Neither	Username	I don't remember	A few times per month or weekly	Find answers to specific questions	Less than once per week	Stack Overflow was slightly faster	60+ minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	No, not at all	Not applicable - I did not use Stack Overflow ...	NaN	34.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
88880	I am not primarily a developer, but I write co...	True	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	2-9 employees	Developer, mobile;Engineering manager;Product ...	20	12	20	Slightly dissatisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	Less than a year ago	Interview with people in peer roles;Interview ...	No	Office environment or company culture;Remote w...	I was preparing for a job search	USD	United States dollar	NaN	NaN	NaN	NaN	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Not enou...	More than half, but not all, the time	Home	NaN	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88881	I am a developer by profession	True	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Austria	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack;Engineer, site reliability	18	17	9	Neither satisfied nor dissatisfied	Neither satisfied nor dissatisfied	Very confident	No	No	I am not interested in new job opportunities	More than 4 years ago	Write any code;Complete a take-home project;In...	No	Office environment or company culture;Diversit...	Something else changed (education, award, medi...	EUR	European Euro	60000.0	Yearly	68745.0	39.00	There is a schedule and/or spec (made by me or...	Distracting work environment;Not enough people...	A few days each month	Office	A little below average	Yes, because I see value in code review	10.0	Yes, it's part of our process	The CTO, CIO, or other management purchase new...	I have little or no influence	Bash/Shell/PowerShell;Go;HTML/CSS;Java;JavaScr...	Bash/Shell/PowerShell;Go;HTML/CSS;JavaScript;P...	PostgreSQL;Redis	Elasticsearch;PostgreSQL;Redis	Docker;Kubernetes;Linux;MacOS;Microsoft Azure;...	Docker;Google Cloud Platform;iOS;Kubernetes;Li...	Django;React.js	Angular/Angular.js;Django;React.js	Ansible	Ansible;Node.js	Emacs;Vim;Visual Studio Code	MacOS	Development;Testing;Production;Outside of work...	Non-currency applications of blockchain	Useful for immutable record keeping outside of...	No	Yes	Yes	NaN	Online	Login	2008	A few times per month or weekly	Find answers to specific questions	More than 10 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	NaN	NaN	37.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
88882	I am a developer by profession	True	Never	OSS is, on average, of LOWER quality than prop...	Employed full-time	Netherlands	Yes, full-time	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Participated in online coding competitions (e....	100 to 499 employees	Academic researcher;Database administrator;Dev...	10	15	8	Slightly dissatisfied	Very dissatisfied	Not at all confident	Yes	Yes	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Complete a take-home project;In...	No	Office environment or company culture;Remote w...	I had a negative experience or interaction at ...	EUR	European Euro	42768.0	Monthly	588012.0	40.00	There is a schedule and/or spec (made by me or...	Distracting work environment;Lack of support f...	Less than once per month / Never	Home	Far above average	Yes, because I see value in code review	5.0	Yes, it's part of our process	Not sure	I have little or no influence	C#;HTML/CSS;Java;JavaScript;PHP;Python	C#;Java;JavaScript;PHP	MariaDB;PostgreSQL	MariaDB;PostgreSQL	Android;Arduino;Linux;Raspberry Pi;WordPress	Android;Arduino;Linux;Raspberry Pi;WordPress	jQuery;React.js	jQuery;React.js	Unity 3D	Unity 3D	Android Studio;IntelliJ;Notepad++;Vim;Visual S...	Linux-based	I do not use containers	Not at all	A passing fad	Yes	Yes	Yes	Instagram	Neither	Username	2009	A few times per week	Find answers to specific questions	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	NaN	NaN	Man	No	Straight / Heterosexual	White or of European descent	Yes	Too long	Easy
88883	I am a developer by profession	True	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Greece	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	20 to 99 employees	Developer, back-end;Developer, desktop or ente...	9	15	5	Slightly dissatisfied	Slightly dissatisfied	Not at all confident	No	No	I’m not actively looking, but I am open to new...	More than 4 years ago	Interview with people in senior / management r...	No	Financial performance or funding status of the...	My job status changed (promotion, new job, etc.)	EUR	European Euro	20000.0	Yearly	22915.0	48.00	There's no schedule or spec; I work on what se...	Inadequate access to necessary tools;Not enoug...	Less than half the time, but at least one day ...	Home	Far above average	No	NaN	Yes, it's part of our process	Not sure	I have little or no influence	Go;HTML/CSS;Java;JavaScript;Other(s):	C++;Go;Java;Rust	Cassandra;MariaDB;PostgreSQL;SQLite	PostgreSQL	Docker;Linux	Docker;Linux	Spring;Vue.js	Spring;Vue.js	NaN	NaN	IntelliJ;Sublime Text	Linux-based	Testing;Production;Outside of work, for person...	Implementing cryptocurrency-based products	A passing fad	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2008	A few times per week	Find answers to specific questions;Learn how t...	1-2 times per week	They were about the same	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers	33.0	Man	No	Straight / Heterosexual	White or of European descent	No	Too short	Easy

88883 rows × 84 columns

In [132]:

df["SocialMedia"]

Out[132]:

Respondent
1          Twitter
2        Instagram
3           Reddit
4           Reddit
5         Facebook
           ...    
88879      YouTube
88880          NaN
88881          NaN
88882    Instagram
88883       Reddit
Name: SocialMedia, Length: 88883, dtype: object

In [133]:

df["SocialMedia"].value_counts()

Out[133]:

Reddit                      14374
YouTube                     13830
WhatsApp                    13347
Facebook                    13178
Twitter                     11398
Instagram                    6261
I don't use social media     5554
LinkedIn                     4501
WeChat 微信                     667
Snapchat                      628
VK ВКонта́кте                 603
Weibo 新浪微博                     56
Youku Tudou 优酷                 21
Hello                          19
Name: SocialMedia, dtype: int64

In [134]:

df["SocialMedia"].value_counts(normalize=True)

Out[134]:

Reddit                      0.170233
YouTube                     0.163791
WhatsApp                    0.158071
Facebook                    0.156069
Twitter                     0.134988
Instagram                   0.074150
I don't use social media    0.065777
LinkedIn                    0.053306
WeChat 微信                   0.007899
Snapchat                    0.007437
VK ВКонта́кте               0.007141
Weibo 新浪微博                  0.000663
Youku Tudou 优酷              0.000249
Hello                       0.000225
Name: SocialMedia, dtype: float64

In [135]:

df["Country"].value_counts()

Out[135]:

United States            20949
India                     9061
Germany                   5866
United Kingdom            5737
Canada                    3395
                         ...  
Sao Tome and Principe        1
North Korea                  1
Timor-Leste                  1
Papua New Guinea             1
Niger                        1
Name: Country, Length: 179, dtype: int64

In [136]:

country_grp = df.groupby(["Country"])

In [137]:

country_grp.get_group("United States")

Out[137]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
4	I am a developer by profession	False	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
13	I am a developer by profession	True	Less than once a month but more than once per ...	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10 to 19 employees	Data or business analyst;Database administrato...	17	11	8	Very satisfied	Very satisfied	NaN	NaN	NaN	I am not interested in new job opportunities	3-4 years ago	Complete a take-home project;Interview with pe...	Yes	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	90000.0	Yearly	90000.0	40.0	There is a schedule and/or spec (made by me or...	Meetings;Non-work commitments (parenting, scho...	All or almost all the time (I'm full-time remote)	Home	A little above average	Yes, because I see value in code review	5.0	No, but I think we should	Developers and management have nearly equal in...	I have a great deal of influence	Bash/Shell/PowerShell;HTML/CSS;JavaScript;PHP;...	Bash/Shell/PowerShell;HTML/CSS;JavaScript;Rust...	Couchbase;DynamoDB;Firebase;MySQL	Firebase;MySQL;Redis	Android;AWS;Docker;IBM Cloud or Watson;iOS;Lin...	Android;AWS;Docker;IBM Cloud or Watson;Linux;S...	Angular/Angular.js;ASP.NET;Express;jQuery;Vue.js	Express;Vue.js	Node.js;Xamarin	Node.js;TensorFlow	Vim;Visual Studio;Visual Studio Code;Xcode	Windows	Development;Testing;Production	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	Yes	Yes	Twitter	In real life (in person)	Username	2011	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Somewhat more welcome now than last year	Tech articles written by other developers;Cour...	28.0	Man	No	Straight / Heterosexual	White or of European descent	Yes	Appropriate in length	Easy
22	I am a developer by profession	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Some college/university study without earning ...	NaN	Taken an online course in programming or softw...	10,000 or more employees	Data or business analyst;Designer;Developer, b...	35	12	18	Slightly satisfied	Very dissatisfied	Somewhat confident	No	No	I’m not actively looking, but I am open to new...	More than 4 years ago	Interview with people in senior / management r...	No	Industry that I'd be working in;Financial perf...	I had a negative experience or interaction at ...	USD	United States dollar	103000.0	Yearly	103000.0	40.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Meeting...	Less than half the time, but at least one day ...	Home	Average	No	NaN	No, but I think we should	The CTO, CIO, or other management purchase new...	I have little or no influence	Bash/Shell/PowerShell;C++;HTML/CSS;JavaScript;...	Bash/Shell/PowerShell;C++;HTML/CSS;JavaScript;...	Elasticsearch;MySQL;Oracle;Redis	Elasticsearch;MySQL;Oracle;Redis	Docker;Linux;Raspberry Pi;Windows	Docker;Linux;Raspberry Pi;Windows	Angular/Angular.js;Ruby on Rails	Angular/Angular.js;Ruby on Rails	Node.js	Node.js	Sublime Text;Visual Studio;Visual Studio Code	Windows	Outside of work, for personal projects	Not at all	NaN	Yes	Yes	Yes	Instagram	Online	Username	I don't remember	Daily or almost daily	Find answers to specific questions	3-5 times per week	Stack Overflow was much faster	0-10 minutes	Yes	A few times per week	Yes	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	47.0	Man	No	Straight / Heterosexual	White or of European descent	Yes	Appropriate in length	Easy
23	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Information systems, information technology, o...	Taken an online course in programming or softw...	10,000 or more employees	Developer, full-stack	3	19	1	Slightly satisfied	Slightly satisfied	Very confident	No	Not sure	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Opportunities for professional development;How...	I was preparing for a job search	USD	United States dollar	69000.0	Yearly	69000.0	40.0	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Non-work...	A few days each month	Office	Average	Yes, because I see value in code review	8.0	Yes, it's part of our process	Developers and management have nearly equal in...	I have little or no influence	Bash/Shell/PowerShell;HTML/CSS;JavaScript;Pyth...	Bash/Shell/PowerShell;Go;HTML/CSS;Java;JavaScr...	Oracle;SQLite	Couchbase;DynamoDB;Elasticsearch;Firebase;Oracle	Docker;Google Cloud Platform	Docker;iOS;Slack	React.js;Ruby on Rails	Express;React.js;Ruby on Rails;Vue.js	NaN	React Native;TensorFlow	Visual Studio Code	MacOS	Development;Testing;Production	NaN	Useful for immutable record keeping outside of...	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Multiple times per day	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	I have never participated in Q&A on Stack Over...	Yes	No, I've heard of them, but I am not part of a...	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	22.0	Man	No	Straight / Heterosexual	Black or of African descent	No	Appropriate in length	Easy
26	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	United States	No	Some college/university study without earning ...	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	10,000 or more employees	Designer;Developer, back-end;Developer, deskto...	12	8	8	Very satisfied	Very satisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	Less than a year ago	Interview with people in peer roles;Interview ...	No	Remote work options;Diversity of the company o...	I was preparing for a job search	USD	United States dollar	114000.0	Yearly	114000.0	40.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Meeting...	Less than half the time, but at least one day ...	Home	Far above average	Yes, because I see value in code review	2.0	Yes, it's not part of our process but the deve...	Developers typically have the most influence o...	I have a great deal of influence	Bash/Shell/PowerShell;C++;C#;HTML/CSS;JavaScri...	C#;HTML/CSS;JavaScript;Objective-C;Ruby;SQL;Sw...	Microsoft SQL Server;MySQL;Redis;SQLite	Microsoft SQL Server;MySQL;Redis;SQLite	AWS;Docker;Linux;MacOS;Microsoft Azure;Windows...	Android;Docker;iOS;Linux;MacOS;Microsoft Azure...	Angular/Angular.js;ASP.NET;Drupal;Express;jQue...	Angular/Angular.js;ASP.NET	.NET;.NET Core;Node.js;Xamarin	.NET;.NET Core;Node.js	Notepad++;Sublime Text;Vim;Visual Studio;Xcode	MacOS	Development;Testing	Not at all	A passing fad	Yes	SIGH	Yes	I don't use social media	In real life (in person)	Username	2008	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	NaN	34.0	Man	No	Gay or Lesbian	NaN	No	Appropriate in length	Easy
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88875	I used to be a developer by profession, but no...	True	Less than once per year	OSS is, on average, of LOWER quality than prop...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	A business discipline (ex. accounting, finance...	Received on-the-job training in software devel...	5,000 to 9,999 employees	Senior executive/VP	13	13	5	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	JavaScript;Python;R;Ruby	NaN	Elasticsearch;Firebase;MongoDB	NaN	AWS;Docker;Google Cloud Platform	NaN	NaN	NaN	Chef;Hadoop;React Native;TensorFlow;Torch/PyTorch	NaN	NaN	NaN	NaN	NaN	Yes	Yes	Yes	LinkedIn	Online	Username	2019	A few times per month or weekly	Learn how to do things I didn’t necessarily lo...	Less than once per week	NaN	NaN	Yes	I have never participated in Q&A on Stack Over...	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	No, not really	Not applicable - I did not use Stack Overflow ...	Tech articles written by other developers;Indu...	35.0	Man	No	NaN	NaN	Yes	Appropriate in length	Easy
88876	I am a developer by profession	True	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Received on-the-job training in software devel...	10,000 or more employees	Developer, back-end;Developer, game or graphics	8	15	2	Very satisfied	Slightly satisfied	Somewhat confident	Yes	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Write code by hand (e.g., on a whiteboard)	No	Office environment or company culture;Remote w...	My job status changed (promotion, new job, etc.)	USD	United States dollar	180000.0	Yearly	180000.0	40.0	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Time spe...	Less than half the time, but at least one day ...	Office	Average	Yes, because I see value in code review	3.0	Yes, it's part of our process	Not sure	I have little or no influence	Bash/Shell/PowerShell;C#;HTML/CSS;Java;Python;...	C#;Java;Kotlin;Scala	DynamoDB;MySQL	DynamoDB	AWS;Linux;MacOS;Windows	Android;AWS;iOS;Linux;MacOS;Windows	NaN	NaN	Apache Spark;Hadoop;.NET Core;Unity 3D	Apache Spark;CryEngine;Hadoop;.NET;.NET Core;T...	IntelliJ;Notepad++;PyCharm;Sublime Text;Vim;Vi...	MacOS	I do not use containers	NaN	A passing fad	Yes	SIGH	Yes	Reddit	Neither	UserID	2011	A few times per month or weekly	Find answers to specific questions	1-2 times per week	Stack Overflow was much faster	60+ minutes	Yes	Less than once per month or monthly	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Yes, somewhat	A lot less welcome now than last year	NaN	23.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
88877	I am a developer by profession	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	500 to 999 employees	Data scientist or machine learning specialist;...	31	18	28	Very satisfied	Very satisfied	Very confident	Yes	Yes	I’m not actively looking, but I am open to new...	More than 4 years ago	Interview with people in senior / management r...	Yes	Industry that I'd be working in;Languages, fra...	I heard about a job opportunity (from a recrui...	USD	United States dollar	239000.0	Weekly	2000000.0	45.0	There is a schedule and/or spec (made by me or...	Meetings;Not enough people for the workload	Less than once per month / Never	Office	Far above average	Yes, because I see value in code review	5.0	No, but I think we should	Developers and management have nearly equal in...	I have some influence	Bash/Shell/PowerShell;C;Clojure;HTML/CSS;Java;...	Bash/Shell/PowerShell;Clojure;HTML/CSS;Java;Ja...	Oracle	Oracle	Docker;iOS;Kubernetes;Linux;Slack;Windows;Othe...	Other(s):	Other(s):	Other(s):	Apache Spark	Apache Spark	Atom;Emacs;IntelliJ;IPython / Jupyter;PyCharm;...	Windows	Development;Testing;Production	Implementing cryptocurrency-based products	An irresponsible use of resources	Yes	Yes	No	Facebook	Online	Screen Name	NaN	A few times per month or weekly	Find answers to specific questions	Less than once per week	Stack Overflow was much faster	11-30 minutes	Yes	I have never participated in Q&A on Stack Over...	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Somewhat more welcome now than last year	Tech articles written by other developers;Cour...	48.0	Man	No	Straight / Heterosexual	South Asian	Yes	Too long	Neither easy nor difficult
88878	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	20 to 99 employees	Developer, back-end;Developer, front-end;Devel...	12	14	3	Very satisfied	Very satisfied	Very confident	Yes	Yes	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	130000.0	Yearly	130000.0	40.0	There is a schedule and/or spec (made by me or...	Non-work commitments (parenting, school work, ...	A few days each month	Office	Far above average	Yes, because I see value in code review	3.0	No, and I'm glad we don't	Developers and management have nearly equal in...	I have some influence	HTML/CSS;JavaScript;Scala;TypeScript	JavaScript;Rust;Scala;TypeScript	PostgreSQL	PostgreSQL	Slack	Slack	React.js;Other(s):	React.js;Other(s):	Node.js	Node.js	IntelliJ;Sublime Text;Visual Studio Code	MacOS	Production	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	Yes	Yes	Twitter	Online	Username	2010	Multiple times per day	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	0-10 minutes	Yes	A few times per week	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	26.0	Man	No	Straight / Heterosexual	South Asian	No	Appropriate in length	Easy
88880	I am not primarily a developer, but I write co...	True	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	2-9 employees	Developer, mobile;Engineering manager;Product ...	20	12	20	Slightly dissatisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	Less than a year ago	Interview with people in peer roles;Interview ...	No	Office environment or company culture;Remote w...	I was preparing for a job search	USD	United States dollar	NaN	NaN	NaN	NaN	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Not enou...	More than half, but not all, the time	Home	NaN	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

20949 rows × 84 columns

In [138]:

filt = df["Country"] == "United States"
df.loc[filt]

Out[138]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
4	I am a developer by profession	False	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
13	I am a developer by profession	True	Less than once a month but more than once per ...	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10 to 19 employees	Data or business analyst;Database administrato...	17	11	8	Very satisfied	Very satisfied	NaN	NaN	NaN	I am not interested in new job opportunities	3-4 years ago	Complete a take-home project;Interview with pe...	Yes	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	90000.0	Yearly	90000.0	40.0	There is a schedule and/or spec (made by me or...	Meetings;Non-work commitments (parenting, scho...	All or almost all the time (I'm full-time remote)	Home	A little above average	Yes, because I see value in code review	5.0	No, but I think we should	Developers and management have nearly equal in...	I have a great deal of influence	Bash/Shell/PowerShell;HTML/CSS;JavaScript;PHP;...	Bash/Shell/PowerShell;HTML/CSS;JavaScript;Rust...	Couchbase;DynamoDB;Firebase;MySQL	Firebase;MySQL;Redis	Android;AWS;Docker;IBM Cloud or Watson;iOS;Lin...	Android;AWS;Docker;IBM Cloud or Watson;Linux;S...	Angular/Angular.js;ASP.NET;Express;jQuery;Vue.js	Express;Vue.js	Node.js;Xamarin	Node.js;TensorFlow	Vim;Visual Studio;Visual Studio Code;Xcode	Windows	Development;Testing;Production	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	Yes	Yes	Twitter	In real life (in person)	Username	2011	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Somewhat more welcome now than last year	Tech articles written by other developers;Cour...	28.0	Man	No	Straight / Heterosexual	White or of European descent	Yes	Appropriate in length	Easy
22	I am a developer by profession	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Some college/university study without earning ...	NaN	Taken an online course in programming or softw...	10,000 or more employees	Data or business analyst;Designer;Developer, b...	35	12	18	Slightly satisfied	Very dissatisfied	Somewhat confident	No	No	I’m not actively looking, but I am open to new...	More than 4 years ago	Interview with people in senior / management r...	No	Industry that I'd be working in;Financial perf...	I had a negative experience or interaction at ...	USD	United States dollar	103000.0	Yearly	103000.0	40.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Meeting...	Less than half the time, but at least one day ...	Home	Average	No	NaN	No, but I think we should	The CTO, CIO, or other management purchase new...	I have little or no influence	Bash/Shell/PowerShell;C++;HTML/CSS;JavaScript;...	Bash/Shell/PowerShell;C++;HTML/CSS;JavaScript;...	Elasticsearch;MySQL;Oracle;Redis	Elasticsearch;MySQL;Oracle;Redis	Docker;Linux;Raspberry Pi;Windows	Docker;Linux;Raspberry Pi;Windows	Angular/Angular.js;Ruby on Rails	Angular/Angular.js;Ruby on Rails	Node.js	Node.js	Sublime Text;Visual Studio;Visual Studio Code	Windows	Outside of work, for personal projects	Not at all	NaN	Yes	Yes	Yes	Instagram	Online	Username	I don't remember	Daily or almost daily	Find answers to specific questions	3-5 times per week	Stack Overflow was much faster	0-10 minutes	Yes	A few times per week	Yes	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	47.0	Man	No	Straight / Heterosexual	White or of European descent	Yes	Appropriate in length	Easy
23	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Information systems, information technology, o...	Taken an online course in programming or softw...	10,000 or more employees	Developer, full-stack	3	19	1	Slightly satisfied	Slightly satisfied	Very confident	No	Not sure	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Opportunities for professional development;How...	I was preparing for a job search	USD	United States dollar	69000.0	Yearly	69000.0	40.0	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Non-work...	A few days each month	Office	Average	Yes, because I see value in code review	8.0	Yes, it's part of our process	Developers and management have nearly equal in...	I have little or no influence	Bash/Shell/PowerShell;HTML/CSS;JavaScript;Pyth...	Bash/Shell/PowerShell;Go;HTML/CSS;Java;JavaScr...	Oracle;SQLite	Couchbase;DynamoDB;Elasticsearch;Firebase;Oracle	Docker;Google Cloud Platform	Docker;iOS;Slack	React.js;Ruby on Rails	Express;React.js;Ruby on Rails;Vue.js	NaN	React Native;TensorFlow	Visual Studio Code	MacOS	Development;Testing;Production	NaN	Useful for immutable record keeping outside of...	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Multiple times per day	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	I have never participated in Q&A on Stack Over...	Yes	No, I've heard of them, but I am not part of a...	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	22.0	Man	No	Straight / Heterosexual	Black or of African descent	No	Appropriate in length	Easy
26	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	United States	No	Some college/university study without earning ...	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	10,000 or more employees	Designer;Developer, back-end;Developer, deskto...	12	8	8	Very satisfied	Very satisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	Less than a year ago	Interview with people in peer roles;Interview ...	No	Remote work options;Diversity of the company o...	I was preparing for a job search	USD	United States dollar	114000.0	Yearly	114000.0	40.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Meeting...	Less than half the time, but at least one day ...	Home	Far above average	Yes, because I see value in code review	2.0	Yes, it's not part of our process but the deve...	Developers typically have the most influence o...	I have a great deal of influence	Bash/Shell/PowerShell;C++;C#;HTML/CSS;JavaScri...	C#;HTML/CSS;JavaScript;Objective-C;Ruby;SQL;Sw...	Microsoft SQL Server;MySQL;Redis;SQLite	Microsoft SQL Server;MySQL;Redis;SQLite	AWS;Docker;Linux;MacOS;Microsoft Azure;Windows...	Android;Docker;iOS;Linux;MacOS;Microsoft Azure...	Angular/Angular.js;ASP.NET;Drupal;Express;jQue...	Angular/Angular.js;ASP.NET	.NET;.NET Core;Node.js;Xamarin	.NET;.NET Core;Node.js	Notepad++;Sublime Text;Vim;Visual Studio;Xcode	MacOS	Development;Testing	Not at all	A passing fad	Yes	SIGH	Yes	I don't use social media	In real life (in person)	Username	2008	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	NaN	34.0	Man	No	Gay or Lesbian	NaN	No	Appropriate in length	Easy
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88875	I used to be a developer by profession, but no...	True	Less than once per year	OSS is, on average, of LOWER quality than prop...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	A business discipline (ex. accounting, finance...	Received on-the-job training in software devel...	5,000 to 9,999 employees	Senior executive/VP	13	13	5	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	JavaScript;Python;R;Ruby	NaN	Elasticsearch;Firebase;MongoDB	NaN	AWS;Docker;Google Cloud Platform	NaN	NaN	NaN	Chef;Hadoop;React Native;TensorFlow;Torch/PyTorch	NaN	NaN	NaN	NaN	NaN	Yes	Yes	Yes	LinkedIn	Online	Username	2019	A few times per month or weekly	Learn how to do things I didn’t necessarily lo...	Less than once per week	NaN	NaN	Yes	I have never participated in Q&A on Stack Over...	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	No, not really	Not applicable - I did not use Stack Overflow ...	Tech articles written by other developers;Indu...	35.0	Man	No	NaN	NaN	Yes	Appropriate in length	Easy
88876	I am a developer by profession	True	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Received on-the-job training in software devel...	10,000 or more employees	Developer, back-end;Developer, game or graphics	8	15	2	Very satisfied	Slightly satisfied	Somewhat confident	Yes	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Write code by hand (e.g., on a whiteboard)	No	Office environment or company culture;Remote w...	My job status changed (promotion, new job, etc.)	USD	United States dollar	180000.0	Yearly	180000.0	40.0	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Time spe...	Less than half the time, but at least one day ...	Office	Average	Yes, because I see value in code review	3.0	Yes, it's part of our process	Not sure	I have little or no influence	Bash/Shell/PowerShell;C#;HTML/CSS;Java;Python;...	C#;Java;Kotlin;Scala	DynamoDB;MySQL	DynamoDB	AWS;Linux;MacOS;Windows	Android;AWS;iOS;Linux;MacOS;Windows	NaN	NaN	Apache Spark;Hadoop;.NET Core;Unity 3D	Apache Spark;CryEngine;Hadoop;.NET;.NET Core;T...	IntelliJ;Notepad++;PyCharm;Sublime Text;Vim;Vi...	MacOS	I do not use containers	NaN	A passing fad	Yes	SIGH	Yes	Reddit	Neither	UserID	2011	A few times per month or weekly	Find answers to specific questions	1-2 times per week	Stack Overflow was much faster	60+ minutes	Yes	Less than once per month or monthly	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Yes, somewhat	A lot less welcome now than last year	NaN	23.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
88877	I am a developer by profession	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	500 to 999 employees	Data scientist or machine learning specialist;...	31	18	28	Very satisfied	Very satisfied	Very confident	Yes	Yes	I’m not actively looking, but I am open to new...	More than 4 years ago	Interview with people in senior / management r...	Yes	Industry that I'd be working in;Languages, fra...	I heard about a job opportunity (from a recrui...	USD	United States dollar	239000.0	Weekly	2000000.0	45.0	There is a schedule and/or spec (made by me or...	Meetings;Not enough people for the workload	Less than once per month / Never	Office	Far above average	Yes, because I see value in code review	5.0	No, but I think we should	Developers and management have nearly equal in...	I have some influence	Bash/Shell/PowerShell;C;Clojure;HTML/CSS;Java;...	Bash/Shell/PowerShell;Clojure;HTML/CSS;Java;Ja...	Oracle	Oracle	Docker;iOS;Kubernetes;Linux;Slack;Windows;Othe...	Other(s):	Other(s):	Other(s):	Apache Spark	Apache Spark	Atom;Emacs;IntelliJ;IPython / Jupyter;PyCharm;...	Windows	Development;Testing;Production	Implementing cryptocurrency-based products	An irresponsible use of resources	Yes	Yes	No	Facebook	Online	Screen Name	NaN	A few times per month or weekly	Find answers to specific questions	Less than once per week	Stack Overflow was much faster	11-30 minutes	Yes	I have never participated in Q&A on Stack Over...	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Somewhat more welcome now than last year	Tech articles written by other developers;Cour...	48.0	Man	No	Straight / Heterosexual	South Asian	Yes	Too long	Neither easy nor difficult
88878	I am a developer by profession	True	Less than once per year	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	20 to 99 employees	Developer, back-end;Developer, front-end;Devel...	12	14	3	Very satisfied	Very satisfied	Very confident	Yes	Yes	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	130000.0	Yearly	130000.0	40.0	There is a schedule and/or spec (made by me or...	Non-work commitments (parenting, school work, ...	A few days each month	Office	Far above average	Yes, because I see value in code review	3.0	No, and I'm glad we don't	Developers and management have nearly equal in...	I have some influence	HTML/CSS;JavaScript;Scala;TypeScript	JavaScript;Rust;Scala;TypeScript	PostgreSQL	PostgreSQL	Slack	Slack	React.js;Other(s):	React.js;Other(s):	Node.js	Node.js	IntelliJ;Sublime Text;Visual Studio Code	MacOS	Production	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	Yes	Yes	Twitter	Online	Username	2010	Multiple times per day	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	0-10 minutes	Yes	A few times per week	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	26.0	Man	No	Straight / Heterosexual	South Asian	No	Appropriate in length	Easy
88880	I am not primarily a developer, but I write co...	True	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	2-9 employees	Developer, mobile;Engineering manager;Product ...	20	12	20	Slightly dissatisfied	Neither satisfied nor dissatisfied	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	Less than a year ago	Interview with people in peer roles;Interview ...	No	Office environment or company culture;Remote w...	I was preparing for a job search	USD	United States dollar	NaN	NaN	NaN	NaN	There is a schedule and/or spec (made by me or...	Distracting work environment;Meetings;Not enou...	More than half, but not all, the time	Home	NaN	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

20949 rows × 84 columns

In [139]:

#we esssentially get the same results

In [140]:

df.loc[filt,"SocialMedia"].value_counts()

Out[140]:

Reddit                      5700
Twitter                     3468
Facebook                    2844
YouTube                     2463
I don't use social media    1851
Instagram                   1652
LinkedIn                    1020
WhatsApp                     609
Snapchat                     326
WeChat 微信                     93
VK ВКонта́кте                  9
Weibo 新浪微博                     8
Hello                          2
Youku Tudou 优酷                 1
Name: SocialMedia, dtype: int64

In [141]:

filt = df["Country"] == "India"
df.loc[filt]

Out[141]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	SalaryUSD	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
8	I code primarily as a hobby	True	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	NaN	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	NaN	Developer, back-end;Engineer, site reliability	8	16	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	Bash/Shell/PowerShell;C;C++;Elixir;Erlang;Go;P...	Cassandra;Elasticsearch;MongoDB;MySQL;Oracle;R...	Cassandra;DynamoDB;Elasticsearch;Firebase;Mong...	AWS;Docker;Heroku;Linux;MacOS;Slack	Android;Arduino;AWS;Docker;Google Cloud Platfo...	Express;Flask;React.js;Spring	Django;Express;Flask;React.js;Vue.js	Hadoop;Node.js;Pandas	Ansible;Apache Spark;Chef;Hadoop;Node.js;Panda...	Atom;IntelliJ;IPython / Jupyter;PyCharm;Visual...	Linux-based	Development;Testing;Production;Outside of work...	NaN	Useful across many domains and could change ma...	Yes	SIGH	Yes	YouTube	In real life (in person)	Handle	2012	A few times per week	Find answers to specific questions;Learn how t...	Less than once per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Yes, definitely	A lot more welcome now than last year	Tech articles written by other developers;Indu...	24.0	Man	No	Straight / Heterosexual	NaN	NaN	Appropriate in length	Neither easy nor difficult
10	I am a developer by profession	True	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	NaN	NaN	10,000 or more employees	Data or business analyst;Data scientist or mac...	12	20	10	Slightly dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	Yes	I’m not actively looking, but I am open to new...	3-4 years ago	NaN	No	Languages, frameworks, and other technologies ...	NaN	INR	Indian rupee	950000.0	Yearly	13293.0	70.0	There's no schedule or spec; I work on what se...	NaN	A few days each month	Home	Far above average	Yes, because I see value in code review	4.0	Yes, it's part of our process	NaN	NaN	C#;Go;JavaScript;Python;R;SQL	C#;Go;JavaScript;Kotlin;Python;R;SQL	Elasticsearch;MongoDB;Microsoft SQL Server;MyS...	Elasticsearch;MongoDB;Microsoft SQL Server	Linux;Windows	Android;Linux;Raspberry Pi;Windows	Angular/Angular.js;ASP.NET;Django;Express;Flas...	Angular/Angular.js;ASP.NET;Django;Express;Flas...	.NET;Node.js;Pandas;Torch/PyTorch	.NET;Node.js;TensorFlow;Torch/PyTorch	Android Studio;Eclipse;IPython / Jupyter;Notep...	Windows	NaN	Not at all	Useful for immutable record keeping outside of...	No	Yes	Yes	YouTube	Neither	Screen Name	NaN	Multiple times per day	Find answers to specific questions;Get a sense...	3-5 times per week	They were about the same	NaN	Yes	A few times per month or weekly	Yes	No, and I don't know what those are	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Tech...	NaN	NaN	NaN	NaN	NaN	Yes	Too long	Difficult
15	I am a student who is learning to code	True	Never	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Student	3	13	NaN	NaN	NaN	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	I've never had a job	NaN	NaN	Industry that I'd be working in;Languages, fra...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Assembly;Bash/Shell/PowerShell;C;C++;HTML/CSS;...	Assembly;Bash/Shell/PowerShell;C;C++;C#;Go;HTM...	MariaDB;MySQL;Oracle;SQLite	MariaDB;MongoDB;Microsoft SQL Server;MySQL;Ora...	Linux;Windows	Android;Google Cloud Platform;iOS;Linux;MacOS;...	NaN	Angular/Angular.js;ASP.NET;Django;Drupal;jQuer...	NaN	.NET;.NET Core;Node.js;TensorFlow;Unity 3D;Unr...	Atom;NetBeans;Notepad++;Sublime Text;Vim	Linux-based	Development	NaN	NaN	Yes	Yes	What?	YouTube	In real life (in person)	NaN	2018	Daily or almost daily	Find answers to specific questions;Learn how t...	More than 10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	20.0	Man	No	NaN	NaN	Yes	Too long	Neither easy nor difficult
50	I am a developer by profession	True	Once a month or more often	OSS is, on average, of LOWER quality than prop...	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Another engineering discipline (ex. civil, ele...	Received on-the-job training in software devel...	10,000 or more employees	Developer, back-end;DevOps specialist	7	15	2	Slightly satisfied	Very satisfied	Very confident	Not sure	Yes	I’m not actively looking, but I am open to new...	1-2 years ago	Write code by hand (e.g., on a whiteboard);Int...	No	Specific department or team I'd be working on;...	I was preparing for a job search	INR	Indian rupee	400000.0	Yearly	5597.0	7.0	There is a schedule and/or spec (made by me or...	Meetings;Time spent commuting	Less than once per month / Never	Other place, such as a coworking space or cafe	Average	No	NaN	Yes, it's not part of our process but the deve...	The CTO, CIO, or other management purchase new...	I have little or no influence	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	HTML/CSS;JavaScript;Python	Elasticsearch;Firebase;MariaDB;MongoDB;MySQL;O...	Firebase;PostgreSQL;Redis;Other(s):	Arduino;AWS;Heroku;Linux;MacOS;Raspberry Pi;Wo...	AWS;Docker;Heroku;Kubernetes;Linux;MacOS;WordP...	Django;Express;Flask;jQuery	Express;Flask;jQuery;React.js;Vue.js	Node.js	Node.js	Notepad++;Visual Studio Code	MacOS	Testing	Not at all	Useful for immutable record keeping outside of...	Yes	Also Yes	What?	YouTube	In real life (in person)	Username	2012	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, definitely	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	23.0	Man	No	NaN	South Asian	No	Too long	Easy
65	I am a developer by profession	True	Never	NaN	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Information systems, information technology, o...	NaN	20 to 99 employees	Developer, front-end;Developer, mobile	2	17	2	Very satisfied	Very satisfied	Very confident	No	Not sure	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Solve a brain-teaser style puzz...	No	Languages, frameworks, and other technologies ...	My job status changed (promotion, new job, etc.)	INR	Indian rupee	NaN	Monthly	NaN	48.0	There's no schedule or spec; I work on what se...	NaN	About half the time	Office	Average	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	Not sure	NaN	Assembly;C;C++;C#;HTML/CSS;Java	Kotlin	Firebase;MySQL;Oracle;SQLite	Firebase;SQLite	Android	Android	ASP.NET	NaN	NaN	NaN	Android Studio;IntelliJ	Linux-based	NaN	NaN	NaN	Yes	Yes	What?	WhatsApp	In real life (in person)	NaN	2017	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	A few times per week	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Not sure	A lot more welcome now than last year	NaN	21.0	Man	No	NaN	NaN	Yes	Appropriate in length	Neither easy nor difficult
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
88829	I am a student who is learning to code	True	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taught yourself a new language, framework, or ...	NaN	Developer, full-stack;Student	3	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Office environment or company culture;Opportun...	I was preparing for a job search	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C++;HTML/CSS;JavaScrip...	Bash/Shell/PowerShell;C;C++;HTML/CSS;JavaScrip...	MySQL;Oracle;PostgreSQL;Redis;SQLite	Couchbase;Elasticsearch;MySQL;PostgreSQL;Redis...	Docker;Linux	Docker;Linux	Django;Express;Flask;jQuery	Angular/Angular.js;Django;Flask;jQuery;React.js	Ansible;Node.js	Ansible;Puppet;React Native	Visual Studio Code	Linux-based	Development;Testing;Production;Outside of work...	NaN	NaN	Yes	Yes	What?	NaN	Online	Username	2016	A few times per week	Find answers to specific questions	6-10 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, definitely	Somewhat more welcome now than last year	Tech articles written by other developers;Indu...	21.0	Man	No	Straight / Heterosexual	South Asian	No	Appropriate in length	Easy
88843	I am a developer by profession	True	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	20 to 99 employees	Developer, back-end;Developer, front-end;Devel...	9	13	2	Very satisfied	Neither satisfied nor dissatisfied	Very confident	Not sure	Yes	I’m not actively looking, but I am open to new...	1-2 years ago	Write any code;Solve a brain-teaser style puzz...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	INR	Indian rupee	NaN	Monthly	NaN	40.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Non-wor...	Less than once per month / Never	Home	A little above average	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	Not sure	I have little or no influence	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	HTML/CSS;JavaScript;Python;SQL;TypeScript	Microsoft SQL Server;MySQL;PostgreSQL	Elasticsearch;MySQL	NaN	NaN	Angular/Angular.js;Django;Express;jQuery	Angular/Angular.js;Django;Express;jQuery;React.js	Node.js	Node.js	IntelliJ;Notepad++;PHPStorm;Visual Studio Code	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	No	Yes	Yes	WhatsApp	In real life (in person)	UserID	2013	Daily or almost daily	Find answers to specific questions;Learn how t...	1-2 times per week	Stack Overflow was slightly faster	0-10 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Somewhat more welcome now than last year	Industry news about technologies you're intere...	22.0	Man	No	Straight / Heterosexual	South Asian	Yes	Appropriate in length	Easy
88856	I am a developer by profession	True	Never	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	Yes, full-time	Master’s degree (MA, MS, M.Eng., MBA, etc.)	Web development or web design	Participated in online coding competitions (e....	5,000 to 9,999 employees	Designer	5	5	2	Neither satisfied nor dissatisfied	Neither satisfied nor dissatisfied	Very confident	NaN	Yes	I’m not actively looking, but I am open to new...	3-4 years ago	Write any code;Write code by hand (e.g., on a ...	No	Specific department or team I'd be working on;...	I heard about a job opportunity (from a recrui...	AOA	Angolan kwanza	NaN	Weekly	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;JavaScript;PHP;TypeScript	HTML/CSS;JavaScript;PHP;TypeScript	NaN	NaN	NaN	NaN	Angular/Angular.js;ASP.NET	Angular/Angular.js;ASP.NET	NaN	NaN	Eclipse;Notepad++;PHPStorm;Sublime Text;Visual...	Windows	Development	Accepting Bitcoin or other coins and tokens as...	Useful across many domains and could change ma...	No	Yes	Yes	NaN	NaN	NaN	2011	I have never visited Stack Overflow (before to...	Find answers to specific questions;Learn how t...	3-5 times per week	NaN	31-60 minutes	NaN	NaN	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	No, not really	Somewhat less welcome now than last year	Tech articles written by other developers;Indu...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
88869	I am a developer by profession	False	Never	OSS is, on average, of LOWER quality than prop...	Employed full-time	India	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Very dissatisfied	Very dissatisfied	Very confident	Yes	Yes	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Remote work options;Opportunities for professi...	I heard about a job opportunity (from a recrui...	INR	Indian rupee	NaN	NaN	NaN	NaN	NaN	Being tasked with non-development work;Distrac...	All or almost all the time (I'm full-time remote)	Home	A little below average	Yes, because I see value in code review	3.0	Yes, it's part of our process	NaN	NaN	HTML/CSS;JavaScript;PHP;SQL	Kotlin;Python;R	MariaDB;MySQL	Elasticsearch;Firebase;MongoDB;Redis	AWS;Heroku;Windows	IBM Cloud or Watson;Kubernetes;Linux	Express;Laravel;React.js;Ruby on Rails	Laravel;React.js;Ruby on Rails	Node.js	Hadoop;Node.js;React Native	NetBeans;Notepad++;PHPStorm	Linux-based	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	No	SIGH	What?	WhatsApp	Online	UserID	2011	Multiple times per day	Find answers to specific questions;Learn how t...	1-2 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Neutral	A lot more welcome now than last year	Tech articles written by other developers	30.0	Man	No	NaN	South Asian	No	Too long	Neither easy nor difficult
88872	I am a developer by profession	True	Less than once a month but more than once per ...	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, back-end;DevOps specialist	7	17	4	Slightly satisfied	Slightly satisfied	Somewhat confident	No	Not sure	I’m not actively looking, but I am open to new...	3-4 years ago	Write code by hand (e.g., on a whiteboard);Sol...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	INR	Indian rupee	NaN	NaN	NaN	NaN	There's no schedule or spec; I work on what se...	Being tasked with non-development work;Meeting...	NaN	Office	A little above average	Yes, because I see value in code review	5.0	Yes, it's part of our process	Developers typically have the most influence o...	I have a great deal of influence	Java;Ruby;SQL	Go;Python;R	Cassandra;MongoDB;MySQL	PostgreSQL;Redis	Docker;Google Cloud Platform;Linux	AWS;Docker;Google Cloud Platform;Kubernetes;Linux	NaN	NaN	NaN	Chef;Puppet	IntelliJ;RubyMine;Vim	Linux-based	Development;Production	NaN	NaN	Yes	Yes	What?	LinkedIn	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Contribute ...	Less than once per week	Stack Overflow was much faster	0-10 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Somewhat more welcome now than last year	Tech articles written by other developers;Cour...	NaN	Man	No	NaN	NaN	Yes	NaN	Neither easy nor difficult

9061 rows × 84 columns

In [142]:

df.loc[filt,"SocialMedia"].value_counts()

Out[142]:

WhatsApp                    2990
YouTube                     1820
LinkedIn                     955
Facebook                     841
Instagram                    822
Twitter                      542
Reddit                       473
I don't use social media     250
Snapchat                      23
WeChat 微信                      5
Hello                          5
VK ВКонта́кте                  4
Youku Tudou 优酷                 2
Weibo 新浪微博                     1
Name: SocialMedia, dtype: int64

In [143]:

country_grp["SocialMedia"].value_counts().head(50)

Out[143]:

Country              SocialMedia             
Afghanistan          Facebook                     15
                     YouTube                       9
                     I don't use social media      6
                     WhatsApp                      4
                     Instagram                     1
                     LinkedIn                      1
                     Twitter                       1
Albania              WhatsApp                     18
                     Facebook                     16
                     Instagram                    13
                     YouTube                      10
                     Twitter                       8
                     LinkedIn                      7
                     Reddit                        6
                     I don't use social media      4
                     Snapchat                      1
                     WeChat 微信                     1
Algeria              YouTube                      42
                     Facebook                     41
                     Twitter                      14
                     LinkedIn                      9
                     I don't use social media      7
                     Instagram                     7
                     Reddit                        2
                     WhatsApp                      2
Andorra              Facebook                      2
                     I don't use social media      2
                     Reddit                        1
                     Snapchat                      1
                     YouTube                       1
Angola               Twitter                       2
                     Facebook                      1
                     YouTube                       1
Antigua and Barbuda  YouTube                       2
                     Facebook                      1
                     Instagram                     1
                     Twitter                       1
                     VK ВКонта́кте                 1
Argentina            WhatsApp                    202
                     Twitter                      80
                     YouTube                      62
                     Instagram                    53
                     Reddit                       50
                     Facebook                     45
                     I don't use social media     22
                     LinkedIn                     16
Armenia              Facebook                     38
                     YouTube                      23
                     LinkedIn                      5
                     Instagram                     3
Name: SocialMedia, dtype: int64

In [144]:

country_grp["SocialMedia"].value_counts().loc["India"] #now we can easily access each country data without changing filters repeatedly

Out[144]:

SocialMedia
WhatsApp                    2990
YouTube                     1820
LinkedIn                     955
Facebook                     841
Instagram                    822
Twitter                      542
Reddit                       473
I don't use social media     250
Snapchat                      23
Hello                          5
WeChat 微信                      5
VK ВКонта́кте                  4
Youku Tudou 优酷                 2
Weibo 新浪微博                     1
Name: SocialMedia, dtype: int64

In [145]:

country_grp["SocialMedia"].value_counts(normalize=True).loc["China"]

Out[145]:

SocialMedia
WeChat 微信                   0.670549
YouTube                     0.088186
Weibo 新浪微博                  0.069884
I don't use social media    0.044925
Twitter                     0.044925
Reddit                      0.019967
LinkedIn                    0.018303
Facebook                    0.013311
Instagram                   0.011647
Youku Tudou 优酷              0.011647
WhatsApp                    0.004992
VK ВКонта́кте               0.001664
Name: SocialMedia, dtype: float64

In [146]:

country_grp["SalaryUSD"].median()

Out[146]:

Country
Afghanistan                               6222.0
Albania                                  10818.0
Algeria                                   7878.0
Andorra                                 160931.0
Angola                                    7764.0
                                          ...   
Venezuela, Bolivarian Republic of...      6384.0
Viet Nam                                 11892.0
Yemen                                    11940.0
Zambia                                    5040.0
Zimbabwe                                 19200.0
Name: SalaryUSD, Length: 179, dtype: float64

In [147]:

country_grp["SalaryUSD"].median().loc["Zimbabwe"]

Out[147]:

19200.0

In [148]:

country_grp["SalaryUSD"].median().loc["India"]

Out[148]:

10080.0

In [149]:

country_grp["SalaryUSD"].agg(["median","mean"]) #aggregate functions passed together

Out[149]:

	median	mean
Country
Afghanistan	6222.0	101953.333333
Albania	10818.0	21833.700000
Algeria	7878.0	34924.047619
Andorra	160931.0	160931.000000
Angola	7764.0	7764.000000
...	...	...
Venezuela, Bolivarian Republic of...	6384.0	14581.627907
Viet Nam	11892.0	17233.436782
Yemen	11940.0	16909.166667
Zambia	5040.0	10075.375000
Zimbabwe	19200.0	34046.666667

179 rows × 2 columns

In [150]:

country_grp["SalaryUSD"].agg(["median","mean"]).loc["Canada"]

Out[150]:

median     68705.000000
mean      134018.564909
Name: Canada, dtype: float64

In [151]:

filt = df["Country"] == "India"
df.loc[filt,"LanguageWorkedWith"].str.contains("Python").sum()

Out[151]:

In [152]:

#country_grp["LanguageWorkedWith"].str.contains("Python").sum().loc["India"]

In [153]:

num_knows_python = country_grp["LanguageWorkedWith"].apply(lambda x: x.str.contains("Python").sum())
num_knows_python

Out[153]:

Country
Afghanistan                              8
Albania                                 23
Algeria                                 40
Andorra                                  0
Angola                                   2
                                        ..
Venezuela, Bolivarian Republic of...    28
Viet Nam                                78
Yemen                                    3
Zambia                                   4
Zimbabwe                                14
Name: LanguageWorkedWith, Length: 179, dtype: int64

In [154]:

total_respondents = country_grp["LanguageWorkedWith"].count()
total_respondents

Out[154]:

Country
Afghanistan                              39
Albania                                  83
Algeria                                 126
Andorra                                   7
Angola                                    4
                                       ... 
Venezuela, Bolivarian Republic of...     87
Viet Nam                                220
Yemen                                    17
Zambia                                   12
Zimbabwe                                 39
Name: LanguageWorkedWith, Length: 179, dtype: int64

In [155]:

num_knows_python/total_respondents

Out[155]:

Country
Afghanistan                             0.205128
Albania                                 0.277108
Algeria                                 0.317460
Andorra                                 0.000000
Angola                                  0.500000
                                          ...   
Venezuela, Bolivarian Republic of...    0.321839
Viet Nam                                0.354545
Yemen                                   0.176471
Zambia                                  0.333333
Zimbabwe                                0.358974
Name: LanguageWorkedWith, Length: 179, dtype: float64

In [156]:

filt = ~df["LanguageWorkedWith"].isna() #to count not NaN

In [157]:

python_df = pd.concat([num_knows_python,df[filt]["Country"].value_counts()], axis = 'columns') #we did not use total_respondents so we get 2 different column names

In [158]:

python_df.rename(columns={"LanguageWorkedWith":"num_knows_python","Country":"total_respondents"},inplace=True)

In [159]:

python_df["pct_knows_python"] = python_df["num_knows_python"]/python_df["total_respondents"]

In [160]:

python_df

Out[160]:

	num_knows_python	total_respondents	pct_knows_python
Afghanistan	8	39.0	0.205128
Albania	23	83.0	0.277108
Algeria	40	126.0	0.317460
Andorra	0	7.0	0.000000
Angola	2	4.0	0.500000
...	...	...	...
Venezuela, Bolivarian Republic of...	28	87.0	0.321839
Viet Nam	78	220.0	0.354545
Yemen	3	17.0	0.176471
Zambia	4	12.0	0.333333
Zimbabwe	14	39.0	0.358974

179 rows × 3 columns

Cleaning Data- Casting Datatypes and Handling Missing Values¶

In [161]:

import numpy as np

In [162]:

people = {"first":["Corey","Jane","John","Chris",np.nan,None,"NA"],
           "last":["Schafer","Doe","Doe","Schafer",np.nan,np.nan,"Missing"],
           "email":["CoreySchafer@gmail.com","JaneDoe@email.com","JohnDoe@email.com",None,np.nan,"Asynchronous","NA"],
            "age":["33","55","63","36",None,None,"Missing"]}

In [163]:

ex_df =pd.DataFrame(people)
ex_df

Out[163]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
3	Chris	Schafer	None	36
4	NaN	NaN	NaN	None
5	None	NaN	Asynchronous	None
6	NA	Missing	NA	Missing

In [164]:

ex_df.dropna()

Out[164]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
6	NA	Missing	NA	Missing

In [165]:

ex_df.dropna(axis="index",how="any") #default arguments-remove the row on finding any NaN values

Out[165]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
6	NA	Missing	NA	Missing

In [166]:

#index can be changed to columns and how can be changed to all

In [167]:

ex_df.dropna(axis="index",how="all") #delete the row if all values are NaN

Out[167]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
3	Chris	Schafer	None	36
5	None	NaN	Asynchronous	None
6	NA	Missing	NA	Missing

In [168]:

ex_df.dropna(axis="columns",how="any") #row 4 had all missing values

Out[168]:


0
1
2
3
4
5
6

In [169]:

ex_df.dropna(axis="index",how="any",subset=["email"]) #if email column has missing values that row will be removed

Out[169]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
5	None	NaN	Asynchronous	None
6	NA	Missing	NA	Missing

In [170]:

ex_df.dropna(axis="index",how="all", subset=["last","email"]) #if both last and email have missing values that row is removed

Out[170]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
3	Chris	Schafer	None	36
5	None	NaN	Asynchronous	None
6	NA	Missing	NA	Missing

In [171]:

ex_df.replace({"NA":np.nan,"Missing":np.nan},inplace=True)
ex_df

Out[171]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
3	Chris	Schafer	None	36
4	NaN	NaN	NaN	None
5	None	NaN	Asynchronous	None
6	NaN	NaN	NaN	NaN

In [172]:

ex_df.dropna()

Out[172]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63

In [173]:

ex_df.isna()

Out[173]:

	first	last	email	age
0	False	False	False	False
1	False	False	False	False
2	False	False	False	False
3	False	False	True	False
4	True	True	True	True
5	True	True	False	True
6	True	True	True	True

In [174]:

ex_df.fillna("MISSING")

Out[174]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33
1	Jane	Doe	JaneDoe@email.com	55
2	John	Doe	JohnDoe@email.com	63
3	Chris	Schafer	MISSING	36
4	MISSING	MISSING	MISSING	MISSING
5	MISSING	MISSING	Asynchronous	MISSING
6	MISSING	MISSING	MISSING	MISSING

In [175]:

ex_df.dtypes

Out[175]:

first    object
last     object
email    object
age      object
dtype: object

In [176]:

#ex_df["age"].mean()

In [177]:

type(np.nan)

Out[177]:

float

In [178]:

#ex_df["age"] = ex_df["age"].astype(int)

In [179]:

ex_df["age"] = ex_df["age"].astype(float)

In [180]:

ex_df

Out[180]:

	first	last	email	age
0	Corey	Schafer	CoreySchafer@gmail.com	33.0
1	Jane	Doe	JaneDoe@email.com	55.0
2	John	Doe	JohnDoe@email.com	63.0
3	Chris	Schafer	None	36.0
4	NaN	NaN	NaN	NaN
5	None	NaN	Asynchronous	NaN
6	NaN	NaN	NaN	NaN

In [181]:

ex_df.dtypes

Out[181]:

first     object
last      object
email     object
age      float64
dtype: object

In [182]:

ex_df["age"].mean()

Out[182]:

46.75

In [183]:

na_vals = ["NA","Missing"]
df = pd.read_csv("survey_results_public.csv", index_col="Respondent",na_values=na_vals)

In [184]:

df.head()

Out[184]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
1	I am a student who is learning to code	Yes	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	4	10	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript;Python	C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL	SQLite	MySQL	MacOS;Windows	Android;Arduino;Windows	Django;Flask	Flask;jQuery	Node.js	Node.js	IntelliJ;Notepad++;PyCharm	Windows	I do not use containers	NaN	NaN	Yes	Fortunately, someone else has that title	Yes	Twitter	Online	Username	2017	A few times per month or weekly	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	31-60 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	I am a student who is learning to code	No	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Developer, desktop or enterprise applications;...	NaN	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Financial performance or funding status of the...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C++;HTML/CSS;Python	C++;HTML/CSS;JavaScript;SQL	NaN	MySQL	Windows	Windows	Django	Django	NaN	NaN	Atom;PyCharm	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	Yes	Instagram	Online	Username	2017	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
3	I am not primarily a developer, but I write co...	Yes	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	Taught yourself a new language, framework, or ...	100 to 499 employees	Designer;Developer, back-end;Developer, front-...	3	22	1	Slightly satisfied	Slightly satisfied	Not at all confident	Not sure	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	THB	Thai baht	23000.0	Monthly	8820.0	40.0	There's no schedule or spec; I work on what se...	Distracting work environment;Inadequate access...	Less than once per month / Never	Home	Average	No	NaN	No, but I think we should	Not sure	I have little or no influence	HTML/CSS	Elixir;HTML/CSS	PostgreSQL	PostgreSQL	NaN	NaN	NaN	Other(s):	NaN	NaN	Vim;Visual Studio Code	Linux-based	I do not use containers	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
4	I am a developer by profession	No	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
5	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Academic researcher;Developer, desktop or ente...	16	14	9	Very dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	No	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	UAH	Ukrainian hryvnia	NaN	NaN	NaN	55.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	A little above average	Yes, because I see value in code review	NaN	Yes, it's part of our process	Not sure	I have little or no influence	C++;HTML/CSS;Java;JavaScript;Python;SQL;VBA	HTML/CSS;Java;JavaScript;SQL;WebAssembly	Couchbase;MongoDB;MySQL;Oracle;PostgreSQL;SQLite	Couchbase;Firebase;MongoDB;MySQL;Oracle;Postgr...	Android;Linux;MacOS;Slack;Windows	Android;Docker;Kubernetes;Linux;Slack	Django;Express;Flask;jQuery;React.js;Spring	Flask;jQuery;React.js;Spring	Cordova;Node.js	Apache Spark;Hadoop;Node.js;React Native	IntelliJ;Notepad++;Vim	Linux-based	Outside of work, for personal projects	Not at all	NaN	Yes	Also Yes	Yes	Facebook	In real life (in person)	Username	I don't remember	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy

In [185]:

df["YearsCode"].head(10)

Out[185]:

Respondent
1       4
2     NaN
3       3
4       3
5      16
6      13
7       6
8       8
9      12
10     12
Name: YearsCode, dtype: object

In [186]:

#df["YearsCode"].mean()

In [187]:

#df["YearsCode"] = df["YearsCode"].astype(float)

In [188]:

df["YearsCode"].unique()

Out[188]:

array(['4', nan, '3', '16', '13', '6', '8', '12', '2', '5', '17', '10',
       '14', '35', '7', 'Less than 1 year', '30', '9', '26', '40', '19',
       '15', '20', '28', '25', '1', '22', '11', '33', '50', '41', '18',
       '34', '24', '23', '42', '27', '21', '36', '32', '39', '38', '31',
       '37', 'More than 50 years', '29', '44', '45', '48', '46', '43',
       '47', '49'], dtype=object)

In [189]:

df["YearsCode"].replace({"Less than 1 year":0,"More than 50 years":51},inplace = True)
df["YearsCode"].unique()

Out[189]:

array(['4', nan, '3', '16', '13', '6', '8', '12', '2', '5', '17', '10',
       '14', '35', '7', 0, '30', '9', '26', '40', '19', '15', '20', '28',
       '25', '1', '22', '11', '33', '50', '41', '18', '34', '24', '23',
       '42', '27', '21', '36', '32', '39', '38', '31', '37', 51, '29',
       '44', '45', '48', '46', '43', '47', '49'], dtype=object)

In [190]:

df["YearsCode"] = df["YearsCode"].astype(float)

In [191]:

df["YearsCode"].mean()

Out[191]:

11.662114216834588

In [192]:

df["YearsCode"].median()

Out[192]:

9.0

Working with Dates and Time Series Data¶

In [193]:

df = pd.read_csv("ETH_1h.csv")
from datetime import datetime

In [194]:

df.head()

Out[194]:

	Date	Symbol	Open	High	Low	Close	Volume
0	2020-03-13 08-PM	ETHUSD	129.94	131.82	126.87	128.71	1940673.93
1	2020-03-13 07-PM	ETHUSD	119.51	132.02	117.10	129.94	7579741.09
2	2020-03-13 06-PM	ETHUSD	124.47	124.85	115.50	119.51	4898735.81
3	2020-03-13 05-PM	ETHUSD	124.08	127.42	121.63	124.47	2753450.92
4	2020-03-13 04-PM	ETHUSD	124.85	129.51	120.17	124.08	4461424.71

In [195]:

#df.loc[0,"Date"].day_name()

In [196]:

#df["Date"] = pd.to_datetime(df["Date"])

In [197]:

df["Date"] = pd.to_datetime(df["Date"], format="%Y-%m-%d %I-%p") #we need to have a proper format for pandas string to convert to datetime

In [198]:

df["Date"]

Out[198]:

0       2020-03-13 20:00:00
1       2020-03-13 19:00:00
2       2020-03-13 18:00:00
3       2020-03-13 17:00:00
4       2020-03-13 16:00:00
                ...        
23669   2017-07-01 15:00:00
23670   2017-07-01 14:00:00
23671   2017-07-01 13:00:00
23672   2017-07-01 12:00:00
23673   2017-07-01 11:00:00
Name: Date, Length: 23674, dtype: datetime64[ns]

In [199]:

df.loc[0,"Date"].day_name()

Out[199]:

'Friday'

In [200]:

d_parser = lambda x: datetime.strptime(x, "%Y-%m-%d %I-%p")
df = pd.read_csv("ETH_1h.csv" , parse_dates=["Date"] , date_parser = d_parser)
df

Out[200]:

	Date	Symbol	Open	High	Low	Close	Volume
0	2020-03-13 20:00:00	ETHUSD	129.94	131.82	126.87	128.71	1940673.93
1	2020-03-13 19:00:00	ETHUSD	119.51	132.02	117.10	129.94	7579741.09
2	2020-03-13 18:00:00	ETHUSD	124.47	124.85	115.50	119.51	4898735.81
3	2020-03-13 17:00:00	ETHUSD	124.08	127.42	121.63	124.47	2753450.92
4	2020-03-13 16:00:00	ETHUSD	124.85	129.51	120.17	124.08	4461424.71
...	...	...	...	...	...	...	...
23669	2017-07-01 15:00:00	ETHUSD	265.74	272.74	265.00	272.57	1500282.55
23670	2017-07-01 14:00:00	ETHUSD	268.79	269.90	265.00	265.74	1702536.85
23671	2017-07-01 13:00:00	ETHUSD	274.83	274.93	265.00	268.79	3010787.99
23672	2017-07-01 12:00:00	ETHUSD	275.01	275.01	271.00	274.83	824362.87
23673	2017-07-01 11:00:00	ETHUSD	279.98	279.99	272.10	275.01	679358.87

23674 rows × 7 columns

In [201]:

df["Date"].dt.day_name()

Out[201]:

0          Friday
1          Friday
2          Friday
3          Friday
4          Friday
           ...   
23669    Saturday
23670    Saturday
23671    Saturday
23672    Saturday
23673    Saturday
Name: Date, Length: 23674, dtype: object

In [202]:

df["DayOfWeek"] = df["Date"].dt.day_name()

In [203]:

df.head()

Out[203]:

	Date	Symbol	Open	High	Low	Close	Volume	DayOfWeek
0	2020-03-13 20:00:00	ETHUSD	129.94	131.82	126.87	128.71	1940673.93	Friday
1	2020-03-13 19:00:00	ETHUSD	119.51	132.02	117.10	129.94	7579741.09	Friday
2	2020-03-13 18:00:00	ETHUSD	124.47	124.85	115.50	119.51	4898735.81	Friday
3	2020-03-13 17:00:00	ETHUSD	124.08	127.42	121.63	124.47	2753450.92	Friday
4	2020-03-13 16:00:00	ETHUSD	124.85	129.51	120.17	124.08	4461424.71	Friday

In [204]:

df["Date"].min()

Out[204]:

Timestamp('2017-07-01 11:00:00')

In [205]:

df["Date"].max()

Out[205]:

Timestamp('2020-03-13 20:00:00')

In [206]:

df["Date"].max()-df["Date"].min()

Out[206]:

Timedelta('986 days 09:00:00')

In [207]:

filt = (df["Date"] >= "2019") & (df["Date"] < "2020")
df.loc[filt]

Out[207]:

	Date	Symbol	Open	High	Low	Close	Volume	DayOfWeek
1749	2019-12-31 23:00:00	ETHUSD	128.33	128.69	128.14	128.54	440678.91	Tuesday
1750	2019-12-31 22:00:00	ETHUSD	128.38	128.69	127.95	128.33	554646.02	Tuesday
1751	2019-12-31 21:00:00	ETHUSD	127.86	128.43	127.72	128.38	350155.69	Tuesday
1752	2019-12-31 20:00:00	ETHUSD	127.84	128.34	127.71	127.86	428183.38	Tuesday
1753	2019-12-31 19:00:00	ETHUSD	128.69	128.69	127.60	127.84	1169847.84	Tuesday
...	...	...	...	...	...	...	...	...
10504	2019-01-01 04:00:00	ETHUSD	130.75	133.96	130.74	131.96	2791135.37	Tuesday
10505	2019-01-01 03:00:00	ETHUSD	130.06	130.79	130.06	130.75	503732.63	Tuesday
10506	2019-01-01 02:00:00	ETHUSD	130.79	130.88	129.55	130.06	838183.43	Tuesday
10507	2019-01-01 01:00:00	ETHUSD	131.62	131.62	130.77	130.79	434917.99	Tuesday
10508	2019-01-01 00:00:00	ETHUSD	130.53	131.91	130.48	131.62	1067136.21	Tuesday

8760 rows × 8 columns

In [208]:

filt = (df["Date"] >= pd.to_datetime("2019")) & (df["Date"] < pd.to_datetime("2020"))
df.loc[filt]

Out[208]:

	Date	Symbol	Open	High	Low	Close	Volume	DayOfWeek
1749	2019-12-31 23:00:00	ETHUSD	128.33	128.69	128.14	128.54	440678.91	Tuesday
1750	2019-12-31 22:00:00	ETHUSD	128.38	128.69	127.95	128.33	554646.02	Tuesday
1751	2019-12-31 21:00:00	ETHUSD	127.86	128.43	127.72	128.38	350155.69	Tuesday
1752	2019-12-31 20:00:00	ETHUSD	127.84	128.34	127.71	127.86	428183.38	Tuesday
1753	2019-12-31 19:00:00	ETHUSD	128.69	128.69	127.60	127.84	1169847.84	Tuesday
...	...	...	...	...	...	...	...	...
10504	2019-01-01 04:00:00	ETHUSD	130.75	133.96	130.74	131.96	2791135.37	Tuesday
10505	2019-01-01 03:00:00	ETHUSD	130.06	130.79	130.06	130.75	503732.63	Tuesday
10506	2019-01-01 02:00:00	ETHUSD	130.79	130.88	129.55	130.06	838183.43	Tuesday
10507	2019-01-01 01:00:00	ETHUSD	131.62	131.62	130.77	130.79	434917.99	Tuesday
10508	2019-01-01 00:00:00	ETHUSD	130.53	131.91	130.48	131.62	1067136.21	Tuesday

8760 rows × 8 columns

In [209]:

df.set_index("Date" , inplace=True)

In [210]:

df.head()

Out[210]:

	Symbol	Open	High	Low	Close	Volume	DayOfWeek
Date
2020-03-13 20:00:00	ETHUSD	129.94	131.82	126.87	128.71	1940673.93	Friday
2020-03-13 19:00:00	ETHUSD	119.51	132.02	117.10	129.94	7579741.09	Friday
2020-03-13 18:00:00	ETHUSD	124.47	124.85	115.50	119.51	4898735.81	Friday
2020-03-13 17:00:00	ETHUSD	124.08	127.42	121.63	124.47	2753450.92	Friday
2020-03-13 16:00:00	ETHUSD	124.85	129.51	120.17	124.08	4461424.71	Friday

In [211]:

df.loc["2019"] #we can access entire years when datetime object is set as index

Out[211]:

	Symbol	Open	High	Low	Close	Volume	DayOfWeek
Date
2019-12-31 23:00:00	ETHUSD	128.33	128.69	128.14	128.54	440678.91	Tuesday
2019-12-31 22:00:00	ETHUSD	128.38	128.69	127.95	128.33	554646.02	Tuesday
2019-12-31 21:00:00	ETHUSD	127.86	128.43	127.72	128.38	350155.69	Tuesday
2019-12-31 20:00:00	ETHUSD	127.84	128.34	127.71	127.86	428183.38	Tuesday
2019-12-31 19:00:00	ETHUSD	128.69	128.69	127.60	127.84	1169847.84	Tuesday
...	...	...	...	...	...	...	...
2019-01-01 04:00:00	ETHUSD	130.75	133.96	130.74	131.96	2791135.37	Tuesday
2019-01-01 03:00:00	ETHUSD	130.06	130.79	130.06	130.75	503732.63	Tuesday
2019-01-01 02:00:00	ETHUSD	130.79	130.88	129.55	130.06	838183.43	Tuesday
2019-01-01 01:00:00	ETHUSD	131.62	131.62	130.77	130.79	434917.99	Tuesday
2019-01-01 00:00:00	ETHUSD	130.53	131.91	130.48	131.62	1067136.21	Tuesday

8760 rows × 7 columns

In [212]:

df.loc["2020-01":"2020-02"]

Out[212]:

	Symbol	Open	High	Low	Close	Volume	DayOfWeek
Date
2020-02-29 23:00:00	ETHUSD	223.35	223.58	216.83	217.31	1927939.88	Saturday
2020-02-29 22:00:00	ETHUSD	223.48	223.59	222.14	223.35	535998.57	Saturday
2020-02-29 21:00:00	ETHUSD	224.63	225.14	222.74	223.48	561158.03	Saturday
2020-02-29 20:00:00	ETHUSD	225.31	225.33	223.50	224.63	511648.65	Saturday
2020-02-29 19:00:00	ETHUSD	225.09	225.85	223.87	225.31	1250856.20	Saturday
...	...	...	...	...	...	...	...
2020-01-01 04:00:00	ETHUSD	129.57	130.00	129.50	129.56	702786.82	Wednesday
2020-01-01 03:00:00	ETHUSD	130.37	130.44	129.38	129.57	496704.23	Wednesday
2020-01-01 02:00:00	ETHUSD	130.14	130.50	129.91	130.37	396315.72	Wednesday
2020-01-01 01:00:00	ETHUSD	128.34	130.14	128.32	130.14	635419.40	Wednesday
2020-01-01 00:00:00	ETHUSD	128.54	128.54	128.12	128.34	245119.91	Wednesday

1440 rows × 7 columns

In [213]:

df.loc["2020-01":"2020-02"]["Close"].mean()

Out[213]:

195.1655902777778

In [214]:

df.loc["2020-01-01"]["High"].max()

Out[214]:

132.68

In [215]:

highs =  df["High"].resample("D").max() #we resample highs on a daily basis

In [216]:

highs["2020-01-01"]

Out[216]:

132.68

In [217]:

%matplotlib inline

In [218]:

highs.plot()

Out[218]:

<AxesSubplot:xlabel='Date'>

In [219]:

df.resample("W").mean()

Out[219]:

	Open	High	Low	Close	Volume
Date
2017-07-02	268.066486	271.124595	264.819730	268.202162	2.185035e+06
2017-07-09	261.337024	262.872917	259.186190	261.062083	1.337349e+06
2017-07-16	196.193214	199.204405	192.722321	195.698393	2.986756e+06
2017-07-23	212.351429	215.779286	209.126310	212.783750	4.298593e+06
2017-07-30	203.496190	205.110357	201.714048	203.309524	1.581729e+06
...	...	...	...	...	...
2020-02-16	255.021667	257.255238	252.679762	255.198452	2.329087e+06
2020-02-23	265.220833	267.263690	262.948512	265.321905	1.826094e+06
2020-03-01	236.720536	238.697500	234.208750	236.373988	2.198762e+06
2020-03-08	229.923571	231.284583	228.373810	229.817619	1.628910e+06
2020-03-15	176.937521	179.979487	172.936239	176.332821	4.259828e+06

142 rows × 5 columns

In [220]:

df.resample("W").agg({"Close":"mean","High":"max","Low":"min", "Volume":"sum"})

Out[220]:

	Close	High	Low	Volume
Date
2017-07-02	268.202162	293.73	253.23	8.084631e+07
2017-07-09	261.062083	285.00	231.25	2.246746e+08
2017-07-16	195.698393	240.33	130.26	5.017750e+08
2017-07-23	212.783750	249.40	153.25	7.221637e+08
2017-07-30	203.309524	229.99	178.03	2.657305e+08
...	...	...	...	...
2020-02-16	255.198452	290.00	216.31	3.912867e+08
2020-02-23	265.321905	287.13	242.36	3.067838e+08
2020-03-01	236.373988	278.13	209.26	3.693920e+08
2020-03-08	229.817619	253.01	196.00	2.736569e+08
2020-03-15	176.332821	208.65	90.00	4.983998e+08

142 rows × 4 columns

Reading and Writing to different sources¶

In [221]:

df = pd.read_csv("survey_results_public.csv",index_col="Respondent")
df.head()

Out[221]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
1	I am a student who is learning to code	Yes	Never	The quality of OSS and closed source software ...	Not employed, and not looking for work	United Kingdom	No	Primary/elementary school	NaN	Taught yourself a new language, framework, or ...	NaN	NaN	4	10	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	HTML/CSS;Java;JavaScript;Python	C;C++;C#;Go;HTML/CSS;Java;JavaScript;Python;SQL	SQLite	MySQL	MacOS;Windows	Android;Arduino;Windows	Django;Flask	Flask;jQuery	Node.js	Node.js	IntelliJ;Notepad++;PyCharm	Windows	I do not use containers	NaN	NaN	Yes	Fortunately, someone else has that title	Yes	Twitter	Online	Username	2017	A few times per month or weekly	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	31-60 minutes	No	NaN	No, I didn't know that Stack Overflow had a jo...	No, and I don't know what those are	Neutral	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	14.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
2	I am a student who is learning to code	No	Less than once per year	The quality of OSS and closed source software ...	Not employed, but looking for work	Bosnia and Herzegovina	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Developer, desktop or enterprise applications;...	NaN	17	NaN	NaN	NaN	NaN	NaN	NaN	I am actively looking for a job	I've never had a job	NaN	NaN	Financial performance or funding status of the...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	C++;HTML/CSS;Python	C++;HTML/CSS;JavaScript;SQL	NaN	MySQL	Windows	Windows	Django	Django	NaN	NaN	Atom;PyCharm	Windows	I do not use containers	NaN	Useful across many domains and could change ma...	Yes	Yes	Yes	Instagram	Online	Username	2017	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was much faster	11-30 minutes	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	19.0	Man	No	Straight / Heterosexual	NaN	No	Appropriate in length	Neither easy nor difficult
3	I am not primarily a developer, but I write co...	Yes	Never	The quality of OSS and closed source software ...	Employed full-time	Thailand	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Web development or web design	Taught yourself a new language, framework, or ...	100 to 499 employees	Designer;Developer, back-end;Developer, front-...	3	22	1	Slightly satisfied	Slightly satisfied	Not at all confident	Not sure	Not sure	I’m not actively looking, but I am open to new...	1-2 years ago	Interview with people in peer roles	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	THB	Thai baht	23000.0	Monthly	8820.0	40.0	There's no schedule or spec; I work on what se...	Distracting work environment;Inadequate access...	Less than once per month / Never	Home	Average	No	NaN	No, but I think we should	Not sure	I have little or no influence	HTML/CSS	Elixir;HTML/CSS	PostgreSQL	PostgreSQL	NaN	NaN	NaN	Other(s):	NaN	NaN	Vim;Visual Studio Code	Linux-based	I do not use containers	NaN	NaN	Yes	Yes	Yes	Reddit	In real life (in person)	Username	2011	A few times per week	Find answers to specific questions;Learn how t...	6-10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Neutral	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	28.0	Man	No	Straight / Heterosexual	NaN	Yes	Appropriate in length	Neither easy nor difficult
4	I am a developer by profession	No	Never	The quality of OSS and closed source software ...	Employed full-time	United States	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	100 to 499 employees	Developer, full-stack	3	16	Less than 1 year	Very satisfied	Slightly satisfied	Very confident	No	Not sure	I am not interested in new job opportunities	Less than a year ago	Write code by hand (e.g., on a whiteboard);Int...	No	Languages, frameworks, and other technologies ...	I was preparing for a job search	USD	United States dollar	61000.0	Yearly	61000.0	80.0	There's no schedule or spec; I work on what se...	NaN	Less than once per month / Never	Home	A little below average	No	NaN	No, but I think we should	Developers typically have the most influence o...	I have little or no influence	C;C++;C#;Python;SQL	C;C#;JavaScript;SQL	MySQL;SQLite	MySQL;SQLite	Linux;Windows	Linux;Windows	NaN	NaN	.NET	.NET	Eclipse;Vim;Visual Studio;Visual Studio Code	Windows	I do not use containers	Not at all	Useful for decentralized currency (i.e., Bitcoin)	Yes	SIGH	Yes	Reddit	In real life (in person)	Username	2014	Daily or almost daily	Find answers to specific questions;Pass the ti...	1-2 times per week	Stack Overflow was much faster	31-60 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	No, not really	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	22.0	Man	No	Straight / Heterosexual	White or of European descent	No	Appropriate in length	Easy
5	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	Ukraine	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taken an online course in programming or softw...	10,000 or more employees	Academic researcher;Developer, desktop or ente...	16	14	9	Very dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	No	I am not interested in new job opportunities	Less than a year ago	Write any code;Write code by hand (e.g., on a ...	No	Industry that I'd be working in;Languages, fra...	I was preparing for a job search	UAH	Ukrainian hryvnia	NaN	NaN	NaN	55.0	There is a schedule and/or spec (made by me or...	Being tasked with non-development work;Inadequ...	A few days each month	Office	A little above average	Yes, because I see value in code review	NaN	Yes, it's part of our process	Not sure	I have little or no influence	C++;HTML/CSS;Java;JavaScript;Python;SQL;VBA	HTML/CSS;Java;JavaScript;SQL;WebAssembly	Couchbase;MongoDB;MySQL;Oracle;PostgreSQL;SQLite	Couchbase;Firebase;MongoDB;MySQL;Oracle;Postgr...	Android;Linux;MacOS;Slack;Windows	Android;Docker;Kubernetes;Linux;Slack	Django;Express;Flask;jQuery;React.js;Spring	Flask;jQuery;React.js;Spring	Cordova;Node.js	Apache Spark;Hadoop;Node.js;React Native	IntelliJ;Notepad++;Vim	Linux-based	Outside of work, for personal projects	Not at all	NaN	Yes	Also Yes	Yes	Facebook	In real life (in person)	Username	I don't remember	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was much faster	NaN	Yes	A few times per month or weekly	No, I knew that Stack Overflow had a job board...	No, I've heard of them, but I am not part of a...	Yes, definitely	Just as welcome now as I felt last year	Tech meetups or events in your area;Courses on...	30.0	Man	No	Straight / Heterosexual	White or of European descent;Multiracial	No	Appropriate in length	Easy

In [222]:

filt = df["Country"] == "India"
india_df = df.loc[filt]
india_df.head()

Out[222]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
8	I code primarily as a hobby	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	NaN	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	NaN	Developer, back-end;Engineer, site reliability	8	16	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	Bash/Shell/PowerShell;C;C++;Elixir;Erlang;Go;P...	Cassandra;Elasticsearch;MongoDB;MySQL;Oracle;R...	Cassandra;DynamoDB;Elasticsearch;Firebase;Mong...	AWS;Docker;Heroku;Linux;MacOS;Slack	Android;Arduino;AWS;Docker;Google Cloud Platfo...	Express;Flask;React.js;Spring	Django;Express;Flask;React.js;Vue.js	Hadoop;Node.js;Pandas	Ansible;Apache Spark;Chef;Hadoop;Node.js;Panda...	Atom;IntelliJ;IPython / Jupyter;PyCharm;Visual...	Linux-based	Development;Testing;Production;Outside of work...	NaN	Useful across many domains and could change ma...	Yes	SIGH	Yes	YouTube	In real life (in person)	Handle	2012	A few times per week	Find answers to specific questions;Learn how t...	Less than once per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Yes, definitely	A lot more welcome now than last year	Tech articles written by other developers;Indu...	24.0	Man	No	Straight / Heterosexual	NaN	NaN	Appropriate in length	Neither easy nor difficult
10	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	NaN	NaN	10,000 or more employees	Data or business analyst;Data scientist or mac...	12	20	10	Slightly dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	Yes	I’m not actively looking, but I am open to new...	3-4 years ago	NaN	No	Languages, frameworks, and other technologies ...	NaN	INR	Indian rupee	950000.0	Yearly	13293.0	70.0	There's no schedule or spec; I work on what se...	NaN	A few days each month	Home	Far above average	Yes, because I see value in code review	4.0	Yes, it's part of our process	NaN	NaN	C#;Go;JavaScript;Python;R;SQL	C#;Go;JavaScript;Kotlin;Python;R;SQL	Elasticsearch;MongoDB;Microsoft SQL Server;MyS...	Elasticsearch;MongoDB;Microsoft SQL Server	Linux;Windows	Android;Linux;Raspberry Pi;Windows	Angular/Angular.js;ASP.NET;Django;Express;Flas...	Angular/Angular.js;ASP.NET;Django;Express;Flas...	.NET;Node.js;Pandas;Torch/PyTorch	.NET;Node.js;TensorFlow;Torch/PyTorch	Android Studio;Eclipse;IPython / Jupyter;Notep...	Windows	NaN	Not at all	Useful for immutable record keeping outside of...	No	Yes	Yes	YouTube	Neither	Screen Name	NaN	Multiple times per day	Find answers to specific questions;Get a sense...	3-5 times per week	They were about the same	NaN	Yes	A few times per month or weekly	Yes	No, and I don't know what those are	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Tech...	NaN	NaN	NaN	NaN	NaN	Yes	Too long	Difficult
15	I am a student who is learning to code	Yes	Never	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Student	3	13	NaN	NaN	NaN	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	I've never had a job	NaN	NaN	Industry that I'd be working in;Languages, fra...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Assembly;Bash/Shell/PowerShell;C;C++;HTML/CSS;...	Assembly;Bash/Shell/PowerShell;C;C++;C#;Go;HTM...	MariaDB;MySQL;Oracle;SQLite	MariaDB;MongoDB;Microsoft SQL Server;MySQL;Ora...	Linux;Windows	Android;Google Cloud Platform;iOS;Linux;MacOS;...	NaN	Angular/Angular.js;ASP.NET;Django;Drupal;jQuer...	NaN	.NET;.NET Core;Node.js;TensorFlow;Unity 3D;Unr...	Atom;NetBeans;Notepad++;Sublime Text;Vim	Linux-based	Development	NaN	NaN	Yes	Yes	What?	YouTube	In real life (in person)	NaN	2018	Daily or almost daily	Find answers to specific questions;Learn how t...	More than 10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	20.0	Man	No	NaN	NaN	Yes	Too long	Neither easy nor difficult
50	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of LOWER quality than prop...	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Another engineering discipline (ex. civil, ele...	Received on-the-job training in software devel...	10,000 or more employees	Developer, back-end;DevOps specialist	7	15	2	Slightly satisfied	Very satisfied	Very confident	Not sure	Yes	I’m not actively looking, but I am open to new...	1-2 years ago	Write code by hand (e.g., on a whiteboard);Int...	No	Specific department or team I'd be working on;...	I was preparing for a job search	INR	Indian rupee	400000.0	Yearly	5597.0	7.0	There is a schedule and/or spec (made by me or...	Meetings;Time spent commuting	Less than once per month / Never	Other place, such as a coworking space or cafe	Average	No	NaN	Yes, it's not part of our process but the deve...	The CTO, CIO, or other management purchase new...	I have little or no influence	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	HTML/CSS;JavaScript;Python	Elasticsearch;Firebase;MariaDB;MongoDB;MySQL;O...	Firebase;PostgreSQL;Redis;Other(s):	Arduino;AWS;Heroku;Linux;MacOS;Raspberry Pi;Wo...	AWS;Docker;Heroku;Kubernetes;Linux;MacOS;WordP...	Django;Express;Flask;jQuery	Express;Flask;jQuery;React.js;Vue.js	Node.js	Node.js	Notepad++;Visual Studio Code	MacOS	Testing	Not at all	Useful for immutable record keeping outside of...	Yes	Also Yes	What?	YouTube	In real life (in person)	Username	2012	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, definitely	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	23.0	Man	No	NaN	South Asian	No	Too long	Easy
65	I am a developer by profession	Yes	Never	NaN	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Information systems, information technology, o...	NaN	20 to 99 employees	Developer, front-end;Developer, mobile	2	17	2	Very satisfied	Very satisfied	Very confident	No	Not sure	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Solve a brain-teaser style puzz...	No	Languages, frameworks, and other technologies ...	My job status changed (promotion, new job, etc.)	INR	Indian rupee	NaN	Monthly	NaN	48.0	There's no schedule or spec; I work on what se...	NaN	About half the time	Office	Average	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	Not sure	NaN	Assembly;C;C++;C#;HTML/CSS;Java	Kotlin	Firebase;MySQL;Oracle;SQLite	Firebase;SQLite	Android	Android	ASP.NET	NaN	NaN	NaN	Android Studio;IntelliJ	Linux-based	NaN	NaN	NaN	Yes	Yes	What?	WhatsApp	In real life (in person)	NaN	2017	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	A few times per week	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Not sure	A lot more welcome now than last year	NaN	21.0	Man	No	NaN	NaN	Yes	Appropriate in length	Neither easy nor difficult

In [223]:

india_df.to_csv("survey_results_india.csv")

In [224]:

india_df.to_csv("survey_results_india.tsv", sep="\t") #tab separated file

In [225]:

#to read tsv use read_csv and add the sep argument as tab

In [226]:

#To work with excel files we need a few packages - xlwt(writing to older format), openpyxl(writing to newer format) and xlrd(reading)

In [227]:

india_df.to_excel("survey_results_india.xlsx")

In [228]:

test = pd.read_excel("survey_results_india.xlsx" , index_col="Respondent")
test.head()

Out[228]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
Respondent
8	I code primarily as a hobby	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	NaN	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	NaN	Developer, back-end;Engineer, site reliability	8	16	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	Bash/Shell/PowerShell;C;C++;Elixir;Erlang;Go;P...	Cassandra;Elasticsearch;MongoDB;MySQL;Oracle;R...	Cassandra;DynamoDB;Elasticsearch;Firebase;Mong...	AWS;Docker;Heroku;Linux;MacOS;Slack	Android;Arduino;AWS;Docker;Google Cloud Platfo...	Express;Flask;React.js;Spring	Django;Express;Flask;React.js;Vue.js	Hadoop;Node.js;Pandas	Ansible;Apache Spark;Chef;Hadoop;Node.js;Panda...	Atom;IntelliJ;IPython / Jupyter;PyCharm;Visual...	Linux-based	Development;Testing;Production;Outside of work...	NaN	Useful across many domains and could change ma...	Yes	SIGH	Yes	YouTube	In real life (in person)	Handle	2012	A few times per week	Find answers to specific questions;Learn how t...	Less than once per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Yes, definitely	A lot more welcome now than last year	Tech articles written by other developers;Indu...	24.0	Man	No	Straight / Heterosexual	NaN	NaN	Appropriate in length	Neither easy nor difficult
10	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	NaN	NaN	10,000 or more employees	Data or business analyst;Data scientist or mac...	12	20	10	Slightly dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	Yes	I’m not actively looking, but I am open to new...	3-4 years ago	NaN	No	Languages, frameworks, and other technologies ...	NaN	INR	Indian rupee	950000.0	Yearly	13293.0	70.0	There's no schedule or spec; I work on what se...	NaN	A few days each month	Home	Far above average	Yes, because I see value in code review	4.0	Yes, it's part of our process	NaN	NaN	C#;Go;JavaScript;Python;R;SQL	C#;Go;JavaScript;Kotlin;Python;R;SQL	Elasticsearch;MongoDB;Microsoft SQL Server;MyS...	Elasticsearch;MongoDB;Microsoft SQL Server	Linux;Windows	Android;Linux;Raspberry Pi;Windows	Angular/Angular.js;ASP.NET;Django;Express;Flas...	Angular/Angular.js;ASP.NET;Django;Express;Flas...	.NET;Node.js;Pandas;Torch/PyTorch	.NET;Node.js;TensorFlow;Torch/PyTorch	Android Studio;Eclipse;IPython / Jupyter;Notep...	Windows	NaN	Not at all	Useful for immutable record keeping outside of...	No	Yes	Yes	YouTube	Neither	Screen Name	NaN	Multiple times per day	Find answers to specific questions;Get a sense...	3-5 times per week	They were about the same	NaN	Yes	A few times per month or weekly	Yes	No, and I don't know what those are	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Tech...	NaN	NaN	NaN	NaN	NaN	Yes	Too long	Difficult
15	I am a student who is learning to code	Yes	Never	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	Yes, full-time	Secondary school (e.g. American high school, G...	NaN	Taken an online course in programming or softw...	NaN	Student	3	13	NaN	NaN	NaN	NaN	NaN	NaN	I’m not actively looking, but I am open to new...	I've never had a job	NaN	NaN	Industry that I'd be working in;Languages, fra...	Something else changed (education, award, medi...	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	Assembly;Bash/Shell/PowerShell;C;C++;HTML/CSS;...	Assembly;Bash/Shell/PowerShell;C;C++;C#;Go;HTM...	MariaDB;MySQL;Oracle;SQLite	MariaDB;MongoDB;Microsoft SQL Server;MySQL;Ora...	Linux;Windows	Android;Google Cloud Platform;iOS;Linux;MacOS;...	NaN	Angular/Angular.js;ASP.NET;Django;Drupal;jQuer...	NaN	.NET;.NET Core;Node.js;TensorFlow;Unity 3D;Unr...	Atom;NetBeans;Notepad++;Sublime Text;Vim	Linux-based	Development	NaN	NaN	Yes	Yes	What?	YouTube	In real life (in person)	NaN	2018	Daily or almost daily	Find answers to specific questions;Learn how t...	More than 10 times per week	They were about the same	NaN	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	20.0	Man	No	NaN	NaN	Yes	Too long	Neither easy nor difficult
50	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of LOWER quality than prop...	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Another engineering discipline (ex. civil, ele...	Received on-the-job training in software devel...	10,000 or more employees	Developer, back-end;DevOps specialist	7	15	2	Slightly satisfied	Very satisfied	Very confident	Not sure	Yes	I’m not actively looking, but I am open to new...	1-2 years ago	Write code by hand (e.g., on a whiteboard);Int...	No	Specific department or team I'd be working on;...	I was preparing for a job search	INR	Indian rupee	400000.0	Yearly	5597.0	7.0	There is a schedule and/or spec (made by me or...	Meetings;Time spent commuting	Less than once per month / Never	Other place, such as a coworking space or cafe	Average	No	NaN	Yes, it's not part of our process but the deve...	The CTO, CIO, or other management purchase new...	I have little or no influence	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	HTML/CSS;JavaScript;Python	Elasticsearch;Firebase;MariaDB;MongoDB;MySQL;O...	Firebase;PostgreSQL;Redis;Other(s):	Arduino;AWS;Heroku;Linux;MacOS;Raspberry Pi;Wo...	AWS;Docker;Heroku;Kubernetes;Linux;MacOS;WordP...	Django;Express;Flask;jQuery	Express;Flask;jQuery;React.js;Vue.js	Node.js	Node.js	Notepad++;Visual Studio Code	MacOS	Testing	Not at all	Useful for immutable record keeping outside of...	Yes	Also Yes	What?	YouTube	In real life (in person)	Username	2012	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, definitely	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	23.0	Man	No	NaN	South Asian	No	Too long	Easy
65	I am a developer by profession	Yes	Never	NaN	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Information systems, information technology, o...	NaN	20 to 99 employees	Developer, front-end;Developer, mobile	2	17	2	Very satisfied	Very satisfied	Very confident	No	Not sure	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Solve a brain-teaser style puzz...	No	Languages, frameworks, and other technologies ...	My job status changed (promotion, new job, etc.)	INR	Indian rupee	NaN	Monthly	NaN	48.0	There's no schedule or spec; I work on what se...	NaN	About half the time	Office	Average	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	Not sure	NaN	Assembly;C;C++;C#;HTML/CSS;Java	Kotlin	Firebase;MySQL;Oracle;SQLite	Firebase;SQLite	Android	Android	ASP.NET	NaN	NaN	NaN	Android Studio;IntelliJ	Linux-based	NaN	NaN	NaN	Yes	Yes	What?	WhatsApp	In real life (in person)	NaN	2017	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	A few times per week	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Not sure	A lot more welcome now than last year	NaN	21.0	Man	No	NaN	NaN	Yes	Appropriate in length	Neither easy nor difficult

In [229]:

india_df.to_json("survey_results_india.json") #nested dictionary (1 column at a time)

In [230]:

india_df.to_json("survey_results_india_2.json" , orient = "records" , lines=True) #non nested dictionary(1 row at a time)

In [231]:

test = pd.read_json("survey_results_india_2.json" , orient = "records" , lines=True)
test.head()

Out[231]:

	MainBranch	Hobbyist	OpenSourcer	OpenSource	Employment	Country	Student	EdLevel	UndergradMajor	EduOther	OrgSize	DevType	YearsCode	Age1stCode	YearsCodePro	CareerSat	JobSat	MgrIdiot	MgrMoney	MgrWant	JobSeek	LastHireDate	LastInt	FizzBuzz	JobFactors	ResumeUpdate	CurrencySymbol	CurrencyDesc	CompTotal	CompFreq	ConvertedComp	WorkWeekHrs	WorkPlan	WorkChallenge	WorkRemote	WorkLoc	ImpSyn	CodeRev	CodeRevHrs	UnitTests	PurchaseHow	PurchaseWhat	LanguageWorkedWith	LanguageDesireNextYear	DatabaseWorkedWith	DatabaseDesireNextYear	PlatformWorkedWith	PlatformDesireNextYear	WebFrameWorkedWith	WebFrameDesireNextYear	MiscTechWorkedWith	MiscTechDesireNextYear	DevEnviron	OpSys	Containers	BlockchainOrg	BlockchainIs	BetterLife	ITperson	OffOn	SocialMedia	Extraversion	ScreenName	SOVisit1st	SOVisitFreq	SOVisitTo	SOFindAnswer	SOTimeSaved	SOHowMuchTime	SOAccount	SOPartFreq	SOJobs	EntTeams	SOComm	WelcomeChange	SONewContent	Age	Gender	Trans	Sexuality	Ethnicity	Dependents	SurveyLength	SurveyEase
0	I code primarily as a hobby	Yes	Less than once per year	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	None	Bachelor’s degree (BA, BS, B.Eng., etc.)	Computer science, computer engineering, or sof...	Taught yourself a new language, framework, or ...	None	Developer, back-end;Engineer, site reliability	8	16	None	None	None	None	None	None	None	None	None	None	None	None	None	None	NaN	None	NaN	NaN	None	None	None	None	None	None	NaN	None	None	None	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	Bash/Shell/PowerShell;C;C++;Elixir;Erlang;Go;P...	Cassandra;Elasticsearch;MongoDB;MySQL;Oracle;R...	Cassandra;DynamoDB;Elasticsearch;Firebase;Mong...	AWS;Docker;Heroku;Linux;MacOS;Slack	Android;Arduino;AWS;Docker;Google Cloud Platfo...	Express;Flask;React.js;Spring	Django;Express;Flask;React.js;Vue.js	Hadoop;Node.js;Pandas	Ansible;Apache Spark;Chef;Hadoop;Node.js;Panda...	Atom;IntelliJ;IPython / Jupyter;PyCharm;Visual...	Linux-based	Development;Testing;Production;Outside of work...	None	Useful across many domains and could change ma...	Yes	SIGH	Yes	YouTube	In real life (in person)	Handle	2012	A few times per week	Find answers to specific questions;Learn how t...	Less than once per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	Yes	No, and I don't know what those are	Yes, definitely	A lot more welcome now than last year	Tech articles written by other developers;Indu...	24.0	Man	No	Straight / Heterosexual	None	None	Appropriate in length	Neither easy nor difficult
1	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of HIGHER quality than pro...	Employed full-time	India	No	Master’s degree (MA, MS, M.Eng., MBA, etc.)	None	None	10,000 or more employees	Data or business analyst;Data scientist or mac...	12	20	10	Slightly dissatisfied	Slightly dissatisfied	Somewhat confident	Yes	Yes	I’m not actively looking, but I am open to new...	3-4 years ago	None	No	Languages, frameworks, and other technologies ...	None	INR	Indian rupee	950000.0	Yearly	13293.0	70.0	There's no schedule or spec; I work on what se...	None	A few days each month	Home	Far above average	Yes, because I see value in code review	4.0	Yes, it's part of our process	None	None	C#;Go;JavaScript;Python;R;SQL	C#;Go;JavaScript;Kotlin;Python;R;SQL	Elasticsearch;MongoDB;Microsoft SQL Server;MyS...	Elasticsearch;MongoDB;Microsoft SQL Server	Linux;Windows	Android;Linux;Raspberry Pi;Windows	Angular/Angular.js;ASP.NET;Django;Express;Flas...	Angular/Angular.js;ASP.NET;Django;Express;Flas...	.NET;Node.js;Pandas;Torch/PyTorch	.NET;Node.js;TensorFlow;Torch/PyTorch	Android Studio;Eclipse;IPython / Jupyter;Notep...	Windows	None	Not at all	Useful for immutable record keeping outside of...	No	Yes	Yes	YouTube	Neither	Screen Name	None	Multiple times per day	Find answers to specific questions;Get a sense...	3-5 times per week	They were about the same	None	Yes	A few times per month or weekly	Yes	No, and I don't know what those are	Yes, somewhat	Somewhat less welcome now than last year	Tech articles written by other developers;Tech...	NaN	None	None	None	None	Yes	Too long	Difficult
2	I am a student who is learning to code	Yes	Never	OSS is, on average, of HIGHER quality than pro...	Not employed, but looking for work	India	Yes, full-time	Secondary school (e.g. American high school, G...	None	Taken an online course in programming or softw...	None	Student	3	13	None	None	None	None	None	None	I’m not actively looking, but I am open to new...	I've never had a job	None	None	Industry that I'd be working in;Languages, fra...	Something else changed (education, award, medi...	None	None	NaN	None	NaN	NaN	None	None	None	None	None	None	NaN	None	None	None	Assembly;Bash/Shell/PowerShell;C;C++;HTML/CSS;...	Assembly;Bash/Shell/PowerShell;C;C++;C#;Go;HTM...	MariaDB;MySQL;Oracle;SQLite	MariaDB;MongoDB;Microsoft SQL Server;MySQL;Ora...	Linux;Windows	Android;Google Cloud Platform;iOS;Linux;MacOS;...	None	Angular/Angular.js;ASP.NET;Django;Drupal;jQuer...	None	.NET;.NET Core;Node.js;TensorFlow;Unity 3D;Unr...	Atom;NetBeans;Notepad++;Sublime Text;Vim	Linux-based	Development	None	None	Yes	Yes	What?	YouTube	In real life (in person)	None	2018	Daily or almost daily	Find answers to specific questions;Learn how t...	More than 10 times per week	They were about the same	None	Yes	Less than once per month or monthly	Yes	No, I've heard of them, but I am not part of a...	Yes, somewhat	Just as welcome now as I felt last year	Tech articles written by other developers;Indu...	20.0	Man	No	None	None	Yes	Too long	Neither easy nor difficult
3	I am a developer by profession	Yes	Once a month or more often	OSS is, on average, of LOWER quality than prop...	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Another engineering discipline (ex. civil, ele...	Received on-the-job training in software devel...	10,000 or more employees	Developer, back-end;DevOps specialist	7	15	2	Slightly satisfied	Very satisfied	Very confident	Not sure	Yes	I’m not actively looking, but I am open to new...	1-2 years ago	Write code by hand (e.g., on a whiteboard);Int...	No	Specific department or team I'd be working on;...	I was preparing for a job search	INR	Indian rupee	400000.0	Yearly	5597.0	7.0	There is a schedule and/or spec (made by me or...	Meetings;Time spent commuting	Less than once per month / Never	Other place, such as a coworking space or cafe	Average	No	NaN	Yes, it's not part of our process but the deve...	The CTO, CIO, or other management purchase new...	I have little or no influence	Bash/Shell/PowerShell;C;C++;HTML/CSS;Java;Java...	HTML/CSS;JavaScript;Python	Elasticsearch;Firebase;MariaDB;MongoDB;MySQL;O...	Firebase;PostgreSQL;Redis;Other(s):	Arduino;AWS;Heroku;Linux;MacOS;Raspberry Pi;Wo...	AWS;Docker;Heroku;Kubernetes;Linux;MacOS;WordP...	Django;Express;Flask;jQuery	Express;Flask;jQuery;React.js;Vue.js	Node.js	Node.js	Notepad++;Visual Studio Code	MacOS	Testing	Not at all	Useful for immutable record keeping outside of...	Yes	Also Yes	What?	YouTube	In real life (in person)	Username	2012	Daily or almost daily	Find answers to specific questions;Learn how t...	3-5 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	Less than once per month or monthly	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Yes, definitely	Just as welcome now as I felt last year	Tech articles written by other developers;Tech...	23.0	Man	No	None	South Asian	No	Too long	Easy
4	I am a developer by profession	Yes	Never	None	Employed full-time	India	No	Bachelor’s degree (BA, BS, B.Eng., etc.)	Information systems, information technology, o...	None	20 to 99 employees	Developer, front-end;Developer, mobile	2	17	2	Very satisfied	Very satisfied	Very confident	No	Not sure	I’m not actively looking, but I am open to new...	Less than a year ago	Write any code;Solve a brain-teaser style puzz...	No	Languages, frameworks, and other technologies ...	My job status changed (promotion, new job, etc.)	INR	Indian rupee	NaN	Monthly	NaN	48.0	There's no schedule or spec; I work on what se...	None	About half the time	Office	Average	Yes, because I see value in code review	NaN	Yes, it's not part of our process but the deve...	Not sure	None	Assembly;C;C++;C#;HTML/CSS;Java	Kotlin	Firebase;MySQL;Oracle;SQLite	Firebase;SQLite	Android	Android	ASP.NET	None	None	None	Android Studio;IntelliJ	Linux-based	None	None	None	Yes	Yes	What?	WhatsApp	In real life (in person)	None	2017	Multiple times per day	Find answers to specific questions	More than 10 times per week	Stack Overflow was slightly faster	11-30 minutes	Yes	A few times per week	No, I knew that Stack Overflow had a job board...	No, and I don't know what those are	Not sure	A lot more welcome now than last year	None	21.0	Man	No	None	None	Yes	Appropriate in length	Neither easy nor difficult

In [ ]: