api-entities/authors/limitations.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "[![GitHub Repository](https://img.shields.io/badge/GitHub-Repository-181717?style=for-the-badge&logo=GitHub&link=https://github.com/Mearman/openalex-docs/tree/main)](https://github.com/Mearman/openalex-docs/tree/main)[![Open in GitHub](https://img.shields.io/badge/Open%20in-GitHub-181717?style=for-the-badge&logo=github&link=https://github.com/Mearman/openalex-docs/blob/main/api-entities/authors/limitations.ipynb)](https://github.com/Mearman/openalex-docs/blob/main/api-entities/authors/limitations.ipynb)[![Open in Colab](https://img.shields.io/badge/Open%20in-Colab-F9AB00?style=for-the-badge&logo=Google%20Colab&link=https://colab.research.google.com/github/Mearman/openalex-docs/blob/main/api-entities/authors/limitations.ipynb)](https://colab.research.google.com/github/Mearman/openalex-docs/blob/main/api-entities/authors/limitations.ipynb)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "%pip install --upgrade \"git+https://github.com/Mearman/openalex-python-pydantic-v1.git\"\n",
        "%pip install pandasai"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "import json\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from openalex_api import Configuration, ApiClient, AutocompleteApi, AuthorsApi, ConceptsApi, FundersApi, InstitutionsApi, PublishersApi, SourcesApi, WorksApi\n",
        "\n",
        "configuration = Configuration(host=\"https://api.openalex.org\")\n",
        "autocomplete_api = AutocompleteApi(ApiClient(configuration))\n",
        "authors_api = AuthorsApi(ApiClient(configuration))\n",
        "concepts_api = ConceptsApi(ApiClient(configuration))\n",
        "funders_api = FundersApi(ApiClient(configuration))\n",
        "institutions_api = InstitutionsApi(ApiClient(configuration))\n",
        "publishers_api = PublishersApi(ApiClient(configuration))\n",
        "sources_api = SourcesApi(ApiClient(configuration))\n",
        "works_api = WorksApi(ApiClient(configuration))\n",
        "\n",
        "from pandasai import SmartDataframe\n",
        "from pandasai.llm import OpenAI"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title  { run: \"auto\", display-mode: \"form\" }\n",
        "openapi_token = \"\" # @param {type:\"string\"}"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Limitations\n",
        "\n",
        "## Works with more than 100 authors are truncated\n",
        "\n",
        "When retrieving a list of works in the API, the `authorships` list within each work will be cut off at 100 authorships objects in order to keep things running well. When this happens the boolean value `is_authors_truncated` will be available and set to `true`. This affects a small portion of OpenAlex, as there are around 35,000 works with more than 100 authors. This limitation does not apply to the [data snapshot](./../../download-all-data/openalex-snapshot.md).\n",
        "\n",
        "* Example list of works with truncated authors\n",
        "  [`https://api.openalex.org/works?filter=authors_count:>100`](https://api.openalex.org/works?filter=authors_count:%3E100)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/works?filter=authors_count:>100\n",
        "filter=\"authors_count:>100\" # @param {type: \"string\"}\n",
        "\n",
        "response = works_api.get_works(\n",
        "\tfilter=filter\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response.results)\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "numeric_df = df[['id', 'display_name'] +\n",
        "\t[col for col in df.columns if df[col].dtype in ['int64', 'float64'] and col != 'relevance_score']]\n",
        "display(numeric_df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "try:\n",
        "\tprompt = \"Visualize this data\" # @param {type:\"string\"}\n",
        "\tSmartDataframe(\n",
        "\t\tnumeric_df,\n",
        "\t\tconfig={\"llm\": (OpenAI(api_token=openapi_token))},\n",
        "\t).chat(prompt)\n",
        "except:\n",
        "\tif not openapi_token:\n",
        "\t\tprint(\"Error: openapi_token not set\")\n",
        "\telse:\n",
        "\t\tprint(\"Error when creating SmartDataframe\")"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To see the full list of authors, go to the individual record for the work, which is never truncated.\n",
        "\n",
        "* Work with all 249 authors available\n",
        "  [`https://api.openalex.org/works/W2168909179`](https://api.openalex.org/works/W2168909179)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/works/W2168909179\n",
        "id=\"W2168909179\" # @param {type: \"string\"}\n",
        "\n",
        "response = works_api.get_work(\n",
        "\tid=id\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response).T.rename(columns=lambda x: x[0]).drop(0).set_index('id')\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This affects filtering as well. So if you filter works using an author ID or ROR, you will not receive works where that author is listed further than 100 places down on the list of authors. We plan to change this in the future, so that filtering works as expected."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}