I’m following the steps mentioned in the repo and I’m running into errors where it is finding lots of modules missing and specifically with tiktoken, it is not able to install it properly. Can anyone help me with this?
Stack Trace:
(chatgpt-retrieval-plugin-py3.12) C:\Windows\System32\chatgpt-retrieval-plugin>pip install tiktoken
Collecting tiktoken
Downloading tiktoken-0.5.1.tar.gz (32 kB)
Installing build dependencies … done
Getting requirements to build wheel … done
Preparing metadata (pyproject.toml) … done
Collecting regex>=2022.1.18 (from tiktoken)
Downloading regex-2023.10.3-cp312-cp312-win_amd64.whl.metadata (41 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 42.0/42.0 kB 503.7 kB/s eta 0:00:00
Requirement already satisfied: requests>=2.26.0 in c:\users\tooba.nazim\appdata\local\pypoetry\cache\virtualenvs\chatgpt-retrieval-plugin-m-xvhbqh-py3.12\lib\site-packages (from tiktoken) (2.31.0)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\tooba.nazim\appdata\local\pypoetry\cache\virtualenvs\chatgpt-retrieval-plugin-m-xvhbqh-py3.12\lib\site-packages (from requests>=2.26.0->tiktoken) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in c:\users\tooba.nazim\appdata\local\pypoetry\cache\virtualenvs\chatgpt-retrieval-plugin-m-xvhbqh-py3.12\lib\site-packages (from requests>=2.26.0->tiktoken) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\tooba.nazim\appdata\local\pypoetry\cache\virtualenvs\chatgpt-retrieval-plugin-m-xvhbqh-py3.12\lib\site-packages (from requests>=2.26.0->tiktoken) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\tooba.nazim\appdata\local\pypoetry\cache\virtualenvs\chatgpt-retrieval-plugin-m-xvhbqh-py3.12\lib\site-packages (from requests>=2.26.0->tiktoken) (2023.7.22)
Downloading regex-2023.10.3-cp312-cp312-win_amd64.whl (268 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 269.0/269.0 kB 2.1 MB/s eta 0:00:00
Building wheels for collected packages: tiktoken
Building wheel for tiktoken (pyproject.toml) … error
error: subprocess-exited-with-error
× Building wheel for tiktoken (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [38 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-312
creating build\lib.win-amd64-cpython-312\tiktoken
copying tiktoken\core.py → build\lib.win-amd64-cpython-312\tiktoken
copying tiktoken\load.py → build\lib.win-amd64-cpython-312\tiktoken
copying tiktoken\model.py → build\lib.win-amd64-cpython-312\tiktoken
copying tiktoken\registry.py → build\lib.win-amd64-cpython-312\tiktoken
copying tiktoken_educational.py → build\lib.win-amd64-cpython-312\tiktoken
copying tiktoken_init_.py → build\lib.win-amd64-cpython-312\tiktoken
creating build\lib.win-amd64-cpython-312\tiktoken_ext
copying tiktoken_ext\openai_public.py → build\lib.win-amd64-cpython-312\tiktoken_ext
running egg_info
writing tiktoken.egg-info\PKG-INFO
writing dependency_links to tiktoken.egg-info\dependency_links.txt
writing requirements to tiktoken.egg-info\requires.txt
writing top-level names to tiktoken.egg-info\top_level.txt
reading manifest file ‘tiktoken.egg-info\SOURCES.txt’
reading manifest template ‘MANIFEST.in’
warning: no files found matching ‘Makefile’
adding license file ‘LICENSE’
writing manifest file ‘tiktoken.egg-info\SOURCES.txt’
copying tiktoken\py.typed → build\lib.win-amd64-cpython-312\tiktoken
running build_ext
running build_rust
error: can’t find Rust compiler
If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
To update pip, run:
pip install --upgrade pip
and then retry package installation.
If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for tiktoken
Failed to build tiktoken
ERROR: Could not build wheels for tiktoken, which is required to install pyproject.toml-based projects