AnalyzeLinuxBugs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "12999d0f-23d3-4dbd-b6f2-cbd3b93857d8",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "# Analyze Linux Bugs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4aea4b0d-4d5a-4281-a97c-db378e7d72d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ea47693-db74-4b88-8cdb-abb6bc4831b3",
   "metadata": {},
   "source": [
    "## Data source\n",
    "- 1000 lasts commits from Linux Kernel GitHub repository\n",
    "- Last commit date -> `Wed Sep 20 12:09:23 2023 -0700)`\n",
    "- Stored as `linux_commits.txt`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0f646e1-780b-4ef7-a7e5-a7303a23444b",
   "metadata": {},
   "source": [
    "## Methodology\n",
    "\n",
    "We will look for the first 100 commits that are a bug fix starting from the last one following these steps:\n",
    "\n",
    "1) Selecting a commit:\n",
    "- I discard those that start with \"Merge ...\"\n",
    "- I check that in the title, in the commit description or in the bug report (mail thread or bugzilla) it is mentioned that it fixes a bug in the software\n",
    "- I check that the change is made in the source code and that it is a change in functionality, excluding variable renaming or comments.\n",
    "\n",
    "2) Extract commit info:\n",
    "- Name/message of the commit (for identification only)\n",
    "- Link to commit\n",
    "- Link to mailing list (if exists)\n",
    "- Link to bugzilla (if exists)\n",
    "- Link to bug-introduction commit (BIC) (if exists)\n",
    "- Location -> Depending on the folder in which the file is changed (e.g. `fs/` => File System)\n",
    "- Regression -> Whether or not it is a regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "460ca1a2-ec80-4800-95ac-6cfb5e0389ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('Bugs_LinuxKernel.csv')  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "084159ac-9391-4eb8-a48f-24b86c564045",
   "metadata": {},
   "source": [
    "## Results\n",
    "\n",
    "After reviewing 203 commits, 100 commits have been found that fixed a bug (Bug-Fixing Commit, BFC)\n",
    "\n",
    "To take an overview of how much information we have been able to collect, we will analyze each of the proposed fields:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d101d651-3bc6-4d62-a44e-c50675c7e7ad",
   "metadata": {},
   "source": [
    "### How many BFCs have a link to the mailing list?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0b4672ea-9b5d-4fe1-bb86-d42c472d1e7b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "49"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df[df[\"MAIL LIST LINK\"]!=\"NONE\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c7c7017-9400-4145-b244-6789c4308dff",
   "metadata": {},
   "source": [
    "**Findings**\n",
    "- Half of the BFCs have an associated mailing list.\n",
    "- Mailing lists are clearly identified, as the format `Link: <URL>` or `Closes: <URL>` is used.\n",
    "- There are cases where there may be more than one mailing list associated with a BFC."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7604797-fb49-40cd-ae9e-f6e2a6f31c94",
   "metadata": {},
   "source": [
    "### How many BFCs have a link to Bugzilla?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7724f05c-4231-4480-92fc-f388c15b9071",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df[df[\"BUGZILLA LINK\"]!=\"NONE\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfe3a5c4-8f7a-44ed-8a24-a28f9f9b803c",
   "metadata": {},
   "source": [
    "**Findings**\n",
    "- The number is very low compared to mailing lists\n",
    "- **Proposal:** Bugzilla bugs could be checked to see if the reported BFCs contain a link to the same Bugzilla issue."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f393c9ff-4f19-4c3a-8c94-05182283a796",
   "metadata": {},
   "source": [
    "### How many BFCs have a link to the BIC?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "3f451f10-474b-45e4-a969-694f06ab7737",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "80"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df[df[\"BIC LINK\"]!=\"NONE\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78bbbb0a-dd7b-42a5-afc0-1150586dd1e6",
   "metadata": {},
   "source": [
    "**Findings**\n",
    "- The numbers are surprisingly high\n",
    "- This is because they follow a format in which they report the BIC in the following format `Fixes: <hash> (<comment>)`\n",
    "- There are cases where more than one BIC is reported.\n",
    "- In many cases they use GitBisect to find the BIC\n",
    "- **Proposal**: It can automatically calculate how much time elapsed between the BIC and the BFC"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e942a197-a707-4efa-a8b8-800f9aaf4005",
   "metadata": {},
   "source": [
    "# Where are the BFCs located?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "7d64ace3-39c0-4f76-9c97-f80b706e90d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LOCATION\n",
       "DRIVER        48\n",
       "FILESYSTEM    24\n",
       "ARCH           7\n",
       "NETWORK        6\n",
       "KERNEL         5\n",
       "INCLUDE        4\n",
       "SCRIPTS        2\n",
       "BLOCK          1\n",
       "IO             1\n",
       "MEMORY         1\n",
       "SECURITY       1\n",
       "Name: ID, dtype: int64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(\"LOCATION\").count().sort_values(\"ID\",ascending=False)['ID']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "111bd144-c14a-45da-a577-615bc954db73",
   "metadata": {},
   "source": [
    "**Findings**\n",
    "- The \"drivers\" module condenses half of the bugs. This is in line with several studies, which show that it is the module where most bugs are fixed."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9303e5dc-ef79-4ca9-a5fa-0f156593bec1",
   "metadata": {},
   "source": [
    "### How many BFCs are detected regressions?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "052577d4-3fe9-46b1-81b0-5e04db6172d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "9"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df[df[\"REGRESSION\"]==\"YES\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4c98354-2586-4bf9-9df5-dbb2bc10829f",
   "metadata": {},
   "source": [
    "**Findings**\n",
    "- To check if it is a regression, it has been verified that either the commit author reports it as such or if it reverses a change of the BIC.\n",
    "- The numbers are not very high, but we may be missing a lot."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01459bd8-aa73-404d-b970-07a3bff6003c",
   "metadata": {},
   "source": [
    "## Other proposals\n",
    "\n",
    "- Check/collect authors of commits (or who signs them)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72b63933-2f93-418b-a8bc-b4e154fc004d",
   "metadata": {},
   "source": [
    "## Examples\n",
    "\n",
    "Below are some examples of commits, either because of their interest in categorization or because of the difficulty they involve:\n",
    "\n",
    "- Commits that looks like a fix, but actually just avoids a warning.\n",
    "    -  https://github.com/torvalds/linux/commit/75b2f7e4c9e0fd750a5a27ca9736d1daa7a3762a\n",
    "    - https://github.com/torvalds/linux/commit/72178d5d1a38dd185d1db15f177f2d122ef10d9b\n",
    "- Commit that rename variables\n",
    "    - https://github.com/torvalds/linux/commit/40d84e198b0ae64df71ac0e70675b16900b90bde \n",
    "- Commit with more than one mailing list\n",
    "    - https://github.com/torvalds/linux/commit/df1c357f25d808e30b216188330e708e09e1a412\n",
    "- Commit with a different mailing list than usual\n",
    "    - https://github.com/torvalds/linux/commit/80cc944eca4f0baa9c381d0706f3160e491437f2\n",
    "- Commit with more than one BIC, one of them being reverted to the BFC and the other having been reverted before.\n",
    "    - https://github.com/torvalds/linux/commit/7a3bc2b3989e05bbaa904a63279049a401491c84\n",
    "- Commits discarded as a fix (but I'm not sure)\n",
    "    - https://github.com/torvalds/linux/commit/e4e14095cc68a2efefba6f77d95efe1137e751d4\n",
    "    - https://github.com/torvalds/linux/commit/06ed09351b67eb1114ae106a87a0ee3ea9adb3db\n",
    "    - https://github.com/torvalds/linux/commit/64be47ba286117ee4e3dd9d064c88ea2913e3269\n",
    "    - https://github.com/torvalds/linux/commit/ec5fa9fcdeca69edf7dab5ca3b2e0ceb1c08fe9a\n",
    "    - https://github.com/torvalds/linux/commit/ab43213e7afd08ac68d4282060bacf309e70fd14\n",
    "    - https://github.com/torvalds/linux/commit/1832403cd41ca6b19b24e9d64f79cb08d920ca44\n",
    "    - https://github.com/torvalds/linux/commit/f5b2c10b57615828b531bb0ae56bd6325a41167e\n",
    "    - https://github.com/torvalds/linux/commit/fc6efed2c728c9c10b058512fc9c1613f870a8e8\n",
    "    - https://github.com/torvalds/linux/commit/97e3c6a853f2af9145daf0c6ca25bcdf55c759d4\n",
    "    - https://github.com/torvalds/linux/commit/9855d60cfc720ff32355484c119acafd3c4dc806\n",
    "    - https://github.com/torvalds/linux/commit/8a81cf96f5510aaf9a65d103f7405079a7b0fcc5\n",
    "- Commits that change/fix only configuration files (Kconfig)\n",
    "    - https://github.com/torvalds/linux/commit/c42d116ccb72b6a33728e2b4b76ab175197ffb07\n",
    "    - https://github.com/torvalds/linux/commit/41425941dfcf47cc6df8e500af6ff16a7be6539f\n",
    "    - https://github.com/torvalds/linux/commit/86e16b87afac20779da1228d690a95c54d7e2ad0\n",
    "    - https://github.com/torvalds/linux/commit/98a15816636044f25be4644db2a3e09fad68aaf7\n",
    "    - https://github.com/torvalds/linux/commit/f6c8a312ef0175ea67a1ace29e1d1e5d470ea45a\n",
    "    - https://github.com/torvalds/linux/commit/b2cacc2e818717545e6d0cc453b72f98249398bf\n",
    "    - https://github.com/torvalds/linux/commit/24dc13f94367edb314b13923818d98dd565edc44\n",
    "- Commit that fix a test\n",
    "    - https://github.com/torvalds/linux/commit/a06023a8f78d3e9e73ca4363ccf3871a06e16ecc"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}