Data Extraction

using Azure Map API

About The Project

This project involved gathering school information from a specific area as requested by a client. I utilized the Azure Map API to efficiently collect the data and deliver it in a well-organized format.

Key aspects of the project:

• Obtaining an Azure API key for accessing the service.
• Implementing Python code to interact with the API.
• Using Jupyter notebooks to streamline testing and debugging of JSON data.

Check out the code on GitHub to see the implementation details.

Final Output Data

Usage

For data retrieval, the project uses a GET HTTP method. Here's an example of how the URL is structured to request data:

To replicate this project, you'll need an Azure API key, which you can obtain here.

For further details on using the Azure API, visit the documentation here.

Survey Monkey

Used a combination of Excel and python (pandas) to clean and prepare the data for the client.

Raw Data:

About The Project

This project focused on refining a large dataset of survey responses from SurveyMonkey. The goal was to process and clean the data to deliver a structured and easily understandable format to the client.

Key aspects of the project:

• Data Extraction and Transformation: Utilized Excel to manipulate and link survey questions with their respective subquestions.
• Data Cleaning: Leveraged Python's pandas library within Jupyter notebooks to further clean and combine the data, transforming it into a more coherent structure.
• Final Output: Exported the cleaned data back into an Excel file for client delivery.

Code Highlights:

• Imported and processed the dataset using pandas and openpyxl.
• Employed data melting and grouping techniques to reorganize and analyze responses.
• Renamed columns for better clarity and exported the final dataset to a new Excel file.

Check out the Jupyter Notebook for details on the data processing steps.

Parental Leave Infographic

Maven Analytics Data challenge

View full Infographic

About this project

The Challenge was to act as an online business journal. My primary goal was to create a nice looking infographic that provided all the data for anyone looking up this Data. They may just be curious, or they may be looking to move companies or countries to look for better benefits.

Cleaning Data

I used Excel to organize and clean the data set, and decide what story I would tell with said Data.

Python

This is my first Challenge here on Maven and I really wanted to make a splash. So I decided to take the Data to the next level using Python.

Using Python and the Google Maps API, I added the location of every company in the list. There were about 4 that could not be located, but enough of them had locations that I could be confident in using the location data to further develop my graphics.

Data Work

I took my new data with locations into Excel for some basic calculations before I took all the data into Tableau. I wanted to make sure I knew what I was looking at, and look for any obvious outliers.

---

Why Did I Use MEDIAN?

Almost every single location and industry had a limited sample set present with one or two outliers that were altering the data.

Let's take the 'Conglomerate' industry, for instance. Out of 10 companies, 2 of them offer exactly 12 weeks and 2 offer over 12 weeks. The rest all offer under 10 weeks leave. Because one company offers 52 weeks, it completely skews the results. The 'Average' for the 'Conglomerate' industry is 16 weeks. The Median is 10.5, which much closer represents the actual data since so many offered under 10 weeks.

I ran these calculations across every location and industry and decided the numbers were getting too skewed by only a small fraction of companies offering either 0 or 52 weeks.

Sample Size

While we are on the topic of sample size, I used the entire table for calculations, but when it came to industry-specific data, I excluded the data if it had under 10 companies.

Why? Well, if an industry shows 4 companies and 4 of them offer 12 weeks paid leave, it leads you to believe that industry as a whole has 100% offering 12+ weeks. The sample size is too small, and several industries were skewed to show 100% paid leave when we know that could not possibly be the case.

---

Insights

If I did my job well, you should be able to quickly gather these insights from the graphic:

• Canada offers the best median leave for both maternity and paternity. With Germany and America offering the least amount.
• The Finance industry has the highest percentage offering 12+ weeks in both paid maternity and paid paternity leave.
• Retail is by far the worst industry when it comes to paid leave.
• Law firms have the highest percentage of over 20 weeks paid maternity leave.
• Paid paternity leave is only offered about 20% of the time, whereas paid maternity leave is offered more than 90% of the time.

---

Thanks for giving me your time!

People illustrations provided by Storyset

You can view the Full project on Maven Analytics

View full Infographic

Natural Language Processing

Review Sentiment Analysis

About this Project

The goal of this project was to analyze product reviews from two different websites to determine the accuracy of review scores using sentiment analysis. By leveraging Selenium and Python, I scraped product reviews from both websites and employed Natural Language Processing (NLP) techniques to assess the sentiment conveyed in these reviews.

Data Collection

Using Selenium, I automated the extraction of product reviews from two distinct websites. This allowed me to gather a substantial amount of review data for subsequent analysis.

Sentiment Analysis

Once the data was collected, I utilized NLP to analyze the sentiment of each review. Specifically, I employed two Hugging Face models—VADER and Roberta—to process and evaluate the sentiment behind the review text.

The analysis revealed a notable discrepancy: one website appeared to artificially inflate its review scores. Despite a broad range of reviews, the scores consistently fell between 4.1 and 5.0, suggesting that the website's scoring system was skewed to present a more positive overall image.

Data Verification

To ensure the accuracy of the sentiment analysis, I manually compared several reviews to their actual text and sentiment scores. This manual check confirmed that the website's review system was indeed skewed, as the lower bound of their 4.1-5.0 scale mis-represented the range of a true 1-10 scale.

Analysis Results

In my analysis, I noticed a clear anomaly where nearly every review from one site (LivingSpaces) was rated between 4.1 and 5.0. This range corresponded to what appeared to be an inflated rating system, skewing the perceived positivity of the reviews.

Proof is in the pudding:

I also noticed LivingSpaces had Identical reviews on completely different SKUs, often totally unrelated items. This was not a one time occurence

_{Identical Rating Examples:}

ID 333: SKU: 241131, Review: The body of the dresser is very sturdy, the drawers are not as sturdy.  I bought this online and was hoping that the drawers would have been better.

ID 283: SKU: 81481, Review: The body of the dresser is very sturdy, the drawers are not as sturdy.  I bought this online and was hoping that the drawers would have been better.

ID 313: SKU: 81478, Review: The body of the dresser is super sturdy, the drawers are not very sturdy.  This is par with how everyone is making furniture anymore. I was hoping for better.

ID 357: SKU: 241125, Review: The body of the dresser is super sturdy, the drawers are not very sturdy.  This is par with how everyone is making furniture anymore. I was hoping for better.

---

Key Insights

• One website's review scores were consistently high, indicating potential artificial inflation.
• Sentiment analysis using VADER and Roberta confirmed discrepancies in the review rating systems.
• Manual verification supported the findings of inflated review scores, as the lower end of the score range represented an exaggerated positive scale.

---

Thank you for exploring this analysis!

Review sentiment analysis powered by Hugging Face NLP models

Full Jupyter notebook and code on GitHub

Employee Demographics

RWFD Employee Demographics Project

About This Project

This project involved creating a comprehensive Demographics Dashboard using HR data from the RWFD Community Project. My goal was to design a visually appealing dashboard that, while appearing straightforward, incorporates a complex array of calculations and custom measures behind the scenes.

Data and Design

Using a combination of Photoshop and Figma, I designed an aesthetically pleasing dashboard interface. The final dashboard was built in Tableau, integrating a wealth of custom calculations and graphics to provide deep insights into employee demographics.

Complex Calculations

This dashboard is one of the most intricate projects I have undertaken, with numerous calculations and data manipulations performed behind the scenes. While many calculations did not end up in the final version, they contributed to the detailed insights provided by the dashboard.

Employee Data

The dashboard includes detailed data on individual employees as well as aggregated employee demographics from 2000 to 2020. This provides a comprehensive view of the workforce over the two-decade period.

---

View the Full Dashboard on Tableau

Retail Price Comparison

Webscraping pricing data using Selenium

About this Project

This project focused on comparing furniture prices across two websites (LivingSpaces and RCWilley) using automated web scraping techniques.
By employing Selenium and Python, The collected data was then stored in a Snowflake database and visualized in a live Power BI dashboard.

Automated Data Collection

I created an automated script with Python to scrape pricing information from the first page of search results for random keywords each day. This script gathered detailed pricing data and review counts, which was subsequently saved to a Snowflake database.

Data Visualization

The data collected from the websites was integrated into a live Power BI dashboard, allowing real-time updates and comparisons. This dashboard provides an accessible interface for viewing and analyzing the pricing trends and review scores across the two websites.

Price Comparison

The core of the analysis involved calculating and comparing the average prices of products across the two websites based on the scraped data. The script calculated the average price and identified both the lowest and highest prices for each website, facilitating a direct comparison between them.

Review Analysis

In addition to pricing, I also observed a peculiar anomaly where one website consistently displayed every item on its first page with at least 5 reviews. Further investigation through sentiment analysis revealed that this site inflates its review scores.

---

View the Power BI Dashboard

*Note: You may be required to sign in to Power BI to view the dashboard.