Error In Installing Spacy En_core_web_lg On Heroku App

Am deploying my ML model on Heroku using Django, I need en_core_web_lg for my application but couldn't install it My requirements.txt is like: .. .. djangorestframework==3.12.2 en-

Solution 1:

I don't know why it is not installing from direct requirements.txt file. But there is another way but that is not right but the work will go on. First you have to remove the package from requirements.txt file for which the error is coming. Then push the app on Heroku, once your app have come to Heroku then wrote this code either from the terminal or from the Heroku dashboard.

if you are using terminal than :

heroku run bash

than run

pip install spacy

and than install which requiremts you want from spacy

python -m spacy download en_core_web_lg

if you are using Heroku dashboard than : first go to your Heroku dashboard click on your app and than at top right click on More and select the Run Console example image than this interface will come up in this you have to click on bash or type bash and run after this you can put same cammand and install. I wish you could solve your problem in this way. (:

Solution 2:

Your error is because there is no pip library named en-core-web-lg. Keep in mind that you cannot put en-core-web-lg in the requirements.txt since it is not a library that you can install using pip. They are models that spacy uses.

The accepted answer is fine. However, if you want a complete automatic deployment, you can also add something like this into your code:

    try:
        nlp = spacy.load("en_core_web_md")
    except: # If not present, we download
        spacy.cli.download("en_core_web_md")
        nlp = spacy.load("en_core_web_md")

This will download the models automatically if they are not present. And, it will not require you to use the Heroku terminal. Only spacy on the requirements.txt file will be necessary.

Finally, keep in mind that spacy models are loaded in memory (RAM). The lg model (about 700MB) is too big to be fitted in a Free, Hobby or Standard 1X dynos (512MB of RAM). So it will fail to load and the dyno will be killed with R15 (https://devcenter.heroku.com/articles/error-codes#r15-memory-quota-vastly-exceeded).

More info about dynos: https://www.heroku.com/dynos