The cost of replacing junior data analysts with GPT-4 is only 0.71%, while replacing it with senior data analysts is 0.45%
You're not mistaken, it's 0.71%, not 71%.
According to the Singapore market, a senior data analyst with an annual salary of 86000-90000 US dollars (600000-630000 RMB) would only need three to four hundred US dollars (over 2000 RMB) to switch to GPT-4.
This conclusion comes from a new paper by Alibaba Dharma Institute and Nanyang Polytechnic University in Singapore, which has been rated as a required reading paper by netizens interested in the fields of AI and data analysis.
Specifically, senior analysts in the conclusion refer to data analysts with years of work experience in the financial industry.
The performance of GPT-4 is comparable to that of a human with 6 years of work experience in most indicators, with lower accuracy than humans, but higher complexity and consistency indicators than humans.
In comparison with another analyst with 5 years of work experience, GPT-4 lost to humans in terms of accuracy of information, aesthetics of charts, and complexity of insights.
If compared to a junior analyst with 2 years of work experience, GPT-4 performs better in accuracy and can complete more work.
But GPT-4 completes all types of tasks much faster than humans.
Assuming that there are 21 working days per month and 8 hours of working time per day, the final conclusion is drawn based on the assumption that the salary is paid at market prices.
What can GPT-4 do as a data analyst
The paper focuses on the following abilities of GPT-4 as a data analyst:
Generate SQL and Python code
Execute code to obtain data and charts
Analyze data and external knowledge sources to draw conclusions
Experiments with 200 samples have shown that GPT-4 is able to understand the meaning of instructions and has some background knowledge of chart types for chart drawing tasks, thus drawing the correct chart.
Most of the charts are clearly visible without any formatting errors. The aesthetic indicators of the icons have a maximum score of 3, with an average GPT-4 score of 2.73.
However, manual inspection can still detect some minor errors, with a maximum score of 1 for chart accuracy indicators and an average GPT-4 score of 0.78.
The paper specifically states that their evaluation criteria are very strict, and any data or labels on the x-axis or y-axis that have errors will result in points being deducted.
For data analysis tasks, GPT-4 achieved an average of full marks in consistency and fluency, verifying that generating fluent and grammatically correct sentences is definitely not a problem for GPT-4.
Interestingly, the accuracy of the data analysis step is much higher than the accuracy of the chart information, indicating that although GPT-4 drew the wrong chart, the analysis yielded the correct conclusion.
In the case study, the research team also summarized three main differences between GPT-4 and human data analysts:
Human analysts can express themselves through personal thoughts and emotions, such as writing 'Surprisingly...' during analysis; Human readers can easily understand whether data meets expectations or is abnormal from such expressions.
Human analysts tend to combine background knowledge
QQ