Exploring_Hacker_News_Posts.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "caa795ed-490d-4d4c-802c-fca13d0274ae",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Exploring Hacker News Posts\n",
    "We'll compare these two types of posts to determine the following:\n",
    "\n",
    "- Do Ask HN or Show HN receive more comments on average?\n",
    "- Do posts created at a certain time receive more comments on average?\n",
    "\n",
    "Dataset can be found [here](https://www.kaggle.com/datasets/hacker-news/hacker-news-posts \"Hacker News Posts | Kaggle\")\n",
    "\n",
    "(Full disclosure this is a guided project.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55c5dcc6-b675-4e49-9877-3d00da53de46",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "62b66fac-a0de-4290-97c5-813a1fd74b65",
   "metadata": {},
   "outputs": [],
   "source": [
    "import datetime as dt\n",
    "from csv import reader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "df31a2d4-bd8a-44ef-93a3-2edca8c6abc5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read in the data and store as a list of lists\n",
    "with open(\"HN_posts_year_to_Sep_26_2016.csv\", encoding=\"utf-8\") as f:\n",
    "    hn = list(reader(f))\n",
    "\n",
    "# Remove the header from the data\n",
    "hn_header, hn_data = hn[0], hn[1:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "af44241e-2616-41b1-8005-db16a44d8583",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Establishing a data summary function as this could be useful later\n",
    "def explore_data(dataset, start, end, rows_and_columns=True):\n",
    "    dataset_slice = dataset[start:end]\n",
    "    for row in dataset_slice:\n",
    "        print(row)\n",
    "    if rows_and_columns:\n",
    "        print(\"Number of rows:\", len(dataset))\n",
    "        print(\"Number of columns:\", len(dataset[0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e9010324-565f-4ad4-a492-ee4055726de4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['id', 'title', 'url', 'num_points', 'num_comments', 'author', 'created_at']\n",
      "['12579008', 'You have two days to comment if you want stem cells to be classified as your own', 'http://www.regulations.gov/document?D=FDA-2015-D-3719-0018', '1', '0', 'altstar', '9/26/2016 3:26']\n",
      "['12579005', 'SQLAR  the SQLite Archiver', 'https://www.sqlite.org/sqlar/doc/trunk/README.md', '1', '0', 'blacksqr', '9/26/2016 3:24']\n",
      "['12578997', 'What if we just printed a flatscreen television on the side of our boxes?', 'https://medium.com/vanmoof/our-secrets-out-f21c1f03fdc8#.ietxmez43', '1', '0', 'pavel_lishin', '9/26/2016 3:19']\n",
      "['12578989', 'algorithmic music', 'http://cacm.acm.org/magazines/2011/7/109891-algorithmic-composition/fulltext', '1', '0', 'poindontcare', '9/26/2016 3:16']\n",
      "['12578979', 'How the Data Vault Enables the Next-Gen Data Warehouse and Data Lake', 'https://www.talend.com/blog/2016/05/12/talend-and-Â\\x93the-data-vaultÂ\\x94', '1', '0', 'markgainor1', '9/26/2016 3:14']\n",
      "Number of rows: 293119\n",
      "Number of columns: 7\n"
     ]
    }
   ],
   "source": [
    "# Print a preview of the data along with the header\n",
    "print(hn_header)\n",
    "explore_data(hn_data, 0, 5, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4de08150-337e-4988-ae3a-f944cf5fcf6b",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Question 1: Do Ask HN or Show HN receive more comments on average?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "b949e59c-4425-4b5c-a741-72be5a99db5b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Ask Posts:  9139\n",
      " Show Posts:  10158\n",
      "Other Posts:  273822\n"
     ]
    }
   ],
   "source": [
    "# Separate posts beginning with \"Ask HN\" and \"Show HN\"\n",
    "ask_posts, show_posts, other_posts = [], [], []\n",
    "for post in hn_data:\n",
    "    title = post[1]\n",
    "    if title.lower().startswith(\"ask hn\"):\n",
    "        ask_posts.append(post)\n",
    "    elif title.lower().startswith(\"show hn\"):\n",
    "        show_posts.append(post)\n",
    "    else:\n",
    "        other_posts.append(post)\n",
    "\n",
    "print(\"  Ask Posts: \", len(ask_posts))\n",
    "print(\" Show Posts: \", len(show_posts))\n",
    "print(\"Other Posts: \", len(other_posts))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "51beeb9b-ee8f-4261-8d36-0dc9bf2b066a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average Comments on Ask Posts: 10.393478498741656\n",
      "Average Comments on Show Posts: 4.886099625910612\n"
     ]
    }
   ],
   "source": [
    "# Find the average number of comments for ask and show post respectively\n",
    "total_ask_comments, total_show_comments = 0, 0\n",
    "\n",
    "for post in ask_posts:\n",
    "    total_ask_comments += int(post[4])\n",
    "\n",
    "for post in show_posts:\n",
    "    total_show_comments += int(post[4])\n",
    "\n",
    "avg_ask_comments = total_ask_comments / len(ask_posts)\n",
    "avg_show_posts = total_show_comments / len(show_posts)\n",
    "\n",
    "print(\"Average Comments on Ask Posts:\", avg_ask_comments)\n",
    "print(\"Average Comments on Show Posts:\", avg_show_posts)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55da72ee-e4c2-4734-8edb-e7fd4cd0e793",
   "metadata": {
    "tags": []
   },
   "source": [
    "On average, \"Ask\" posts receive more than twice as many comments as \"Show\" posts.\n",
    "With this in mind, \"Ask\" posts will be our focus for the rest of this analysis.\n",
    "\n",
    "Next, we'll determine if \"Ask\" posts created at a certain time are more likely to attract comments. We'll use the following steps to perform this analysis:\n",
    "\n",
    "1. Calculate the amount of ask posts created in each hour of the day, along with the number of comments received.\n",
    "1. Calculate the average number of comments ask posts receive by hour created."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "215bc746-5816-4277-986d-7b36d68f6373",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Question 2: Do posts created at a certain time receive more comments on average?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "655ee811-65eb-435d-b79d-30df85a80ca9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['9/26/2016 2:53', 7]\n",
      "['9/26/2016 1:17', 3]\n",
      "['9/25/2016 22:57', 0]\n",
      "['9/25/2016 22:48', 3]\n",
      "['9/25/2016 21:50', 2]\n",
      "Number of rows: 9139\n",
      "Number of columns: 2\n"
     ]
    }
   ],
   "source": [
    "# Create a new list with a column including the time a post was created and a column\n",
    "# including the number of comments that post recieved\n",
    "result_list = [[post[6], int(post[4])] for post in ask_posts]\n",
    "\n",
    "explore_data(result_list, 0, 5, True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "08b90da1-986c-4869-b3f3-295c64819c83",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'02': 269, '01': 282, '22': 383, '21': 518, '19': 552, '17': 587, '15': 646, '14': 513, '13': 444, '11': 312, '10': 282, '09': 222, '07': 226, '03': 271, '23': 343, '20': 510, '16': 579, '08': 257, '00': 301, '18': 614, '12': 342, '04': 243, '06': 234, '05': 209}\n",
      "{'02': 2996, '01': 2089, '22': 3372, '21': 4500, '19': 3954, '17': 5547, '15': 18525, '14': 4972, '13': 7245, '11': 2797, '10': 3013, '09': 1477, '07': 1585, '03': 2154, '23': 2297, '20': 4462, '16': 4466, '08': 2362, '00': 2277, '18': 4877, '12': 4234, '04': 2360, '06': 1587, '05': 1838}\n"
     ]
    }
   ],
   "source": [
    "# Calculate the number of ask posts created per hour and the total number of comments\n",
    "counts_by_hour, comments_by_hour = {}, {}\n",
    "for result in result_list:\n",
    "    date = dt.datetime.strptime(result[0], \"%m/%d/%Y %H:%M\")\n",
    "    hour = date.strftime(\"%H\")\n",
    "    if hour in counts_by_hour:\n",
    "        counts_by_hour[hour] += 1\n",
    "        comments_by_hour[hour] += result[1]\n",
    "    else:\n",
    "        counts_by_hour[hour] = 1\n",
    "        comments_by_hour[hour] = result[1]\n",
    "\n",
    "print(counts_by_hour)\n",
    "print(comments_by_hour)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "661e2163-50ab-405c-b7ad-57e93976f479",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['00', 7.5647840531561465]\n",
      "['01', 7.407801418439717]\n",
      "['02', 11.137546468401487]\n",
      "['03', 7.948339483394834]\n",
      "['04', 9.7119341563786]\n",
      "['05', 8.794258373205741]\n",
      "['06', 6.782051282051282]\n",
      "['07', 7.013274336283186]\n",
      "['08', 9.190661478599221]\n",
      "['09', 6.653153153153153]\n",
      "['10', 10.684397163120567]\n",
      "['11', 8.96474358974359]\n",
      "['12', 12.380116959064328]\n",
      "['13', 16.31756756756757]\n",
      "['14', 9.692007797270955]\n",
      "['15', 28.676470588235293]\n",
      "['16', 7.713298791018998]\n",
      "['17', 9.449744463373083]\n",
      "['18', 7.94299674267101]\n",
      "['19', 7.163043478260869]\n",
      "['20', 8.749019607843136]\n",
      "['21', 8.687258687258687]\n",
      "['22', 8.804177545691905]\n",
      "['23', 6.696793002915452]\n"
     ]
    }
   ],
   "source": [
    "# Calculate the average number of comments on posts created during each hour of the day\n",
    "avg_by_hour = []\n",
    "for hour in counts_by_hour:\n",
    "    avg_by_hour.append([hour, comments_by_hour[hour] / counts_by_hour[hour]])\n",
    "\n",
    "print(*sorted(avg_by_hour), sep = \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "921baa1c-1be9-4685-b604-9d5665285007",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['15', 28.676470588235293]\n",
      "['13', 16.31756756756757]\n",
      "['12', 12.380116959064328]\n",
      "['02', 11.137546468401487]\n",
      "['10', 10.684397163120567]\n",
      "['04', 9.7119341563786]\n",
      "['14', 9.692007797270955]\n",
      "['17', 9.449744463373083]\n",
      "['08', 9.190661478599221]\n",
      "['11', 8.96474358974359]\n",
      "['22', 8.804177545691905]\n",
      "['05', 8.794258373205741]\n",
      "['20', 8.749019607843136]\n",
      "['21', 8.687258687258687]\n",
      "['03', 7.948339483394834]\n",
      "['18', 7.94299674267101]\n",
      "['16', 7.713298791018998]\n",
      "['00', 7.5647840531561465]\n",
      "['01', 7.407801418439717]\n",
      "['19', 7.163043478260869]\n",
      "['07', 7.013274336283186]\n",
      "['06', 6.782051282051282]\n",
      "['23', 6.696793002915452]\n",
      "['09', 6.653153153153153]\n"
     ]
    }
   ],
   "source": [
    "# Sort the list by average in descending order\n",
    "avg_by_hour_sorted = sorted(avg_by_hour, key=lambda x:x[1], reverse=True)\n",
    "\n",
    "print(*avg_by_hour_sorted, sep = \"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "f2b76d8f-c58c-4c7e-af8e-44b409274c84",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Top 5 Hours for Ask Posts Comments\n",
      "\n",
      "15:00 Eastern Time 28.68 average comments per post\n",
      "13:00 Eastern Time 16.32 average comments per post\n",
      "12:00 Eastern Time 12.38 average comments per post\n",
      "02:00 Eastern Time 11.14 average comments per post\n",
      "10:00 Eastern Time 10.68 average comments per post\n"
     ]
    }
   ],
   "source": [
    "# Print our results\n",
    "print(\"Top 5 Hours for Ask Posts Comments\\n\")\n",
    "for hour in avg_by_hour_sorted[:5]:\n",
    "    print(\"{}:00 Eastern Time {:.2f} average comments per post\".format(hour[0], hour[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c8c9ecc-de7b-436f-b115-e900e70bd23e",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Conclusion\n",
    "Thus far this analysis suggests:\n",
    "- \"Ask\" posts recieve more comments than \"Show\" posts on average\n",
    "- \"Ask\" posts created from 1500 to 1600 Eastern Time recieve more comments on average than \"Ask\" posts created during the other hours of the day\n",
    "\n",
    "These insights taken together suggest an \"Ask\" post created from 1500 to 1600 Eastern Time are more likely to elicit engagement then \"Ask\" and \"Show\" posts created at other hours of the day.\n",
    "For a brand looking to maximize engagement this is an actionable insight."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b69148d-d269-4e71-8ffd-b7164bf8b31b",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Potential Next Steps\n",
    "- Determine if show or ask posts receive more points on average.\n",
    "- Determine if posts created at a certain time are more likely to receive more points."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}