Hey @deepscreener. I have a Mac M1 Pro and it works fine for me. It seems that it has to do with your tiktoken version (it states that you have installed the Intelβs version). Maybe reinstalling tiktoken?
Here is my M1 system information and a Tiktoken example, hope it helps you @deepscreener
Mac M1 Info:
MacStudio$ system_profiler SPSoftwareDataType SPHardwareDataType
Software:
System Software Overview:
System Version: macOS 13.2.1 (22D68)
Kernel Version: Darwin 22.3.0
Boot Volume: Macintosh HD
Boot Mode: Normal
Computer Name: MacStudio
User Name: Tim Bass (tim)
Secure Virtual Memory: Enabled
System Integrity Protection: Enabled
Time since boot: 1 day, 15 hours, 8 minutes
Hardware:
Hardware Overview:
Model Name: Mac Studio
Model Identifier: Mac13,1
Model Number: Z14J0006CTH/A
Chip: Apple M1 Max
Total Number of Cores: 10 (8 performance and 2 efficiency)
Memory: 32 GB
System Firmware Version: 8419.80.7
OS Loader Version: 8419.80.7
Serial Number (system): XFWHDJ4HVK
Hardware UUID: 0F7157D3-3F8F-5BB7-A101-1A4308FC2202
Provisioning UDID: 00006001-000241AC3C41801E
Activation Lock Status: Enabled
I just double checked this and found this Rust dependency on GitHub, I was not aware of:
Rust Info
MacStudio:$ brew info rust
==> rust: stable 1.67.1 (bottled), HEAD
Safe, concurrent, practical language
Not sure if this helps or not.
Shout out to @RonaldGRuckus for pointing this out. I did not realize this dependency existed since I donβt use Rust these days and forgot Rust was installed on my Mac.
Final Note.
I uninstalled Rust on my Mac, as follows:
brew uninstall rust
Confirm
MacStudio:scripts tim$ brew info rust
==> rust: stable 1.67.1 (bottled), HEAD
Safe, concurrent, practical language
https://www.rust-lang.org/
Not installed