Hello,
I've previously been able to get models to pass tests, but tried again a few hours ago with a new model and am getting a weird (lack of) result from dynalab-cli test: the last few lines of output look like this:
Step 23/25 : ENV model_name=${model_name}
---> Running in 1becff8897f9
Removing intermediate container 1becff8897f9
---> f60d205fd0a2
Step 24/25 : ENV PYTHONIOENCODING=UTF-8
---> Running in aabfc23ddb47
Removing intermediate container aabfc23ddb47
---> 428861ec2ac0
Step 25/25 : ENTRYPOINT /usr/local/bin/dev-entrypoint.sh ${model_name}
---> Running in 6f2a6d4a201b
Removing intermediate container 6f2a6d4a201b
---> 3d77ba06f222
Successfully built 3d77ba06f222
Successfully tagged adapter-ensemble:latest
It's been hanging like this for a couple of hours. I tried running docker run adapter-ensemble and got this:
Running integrated test for model adapter-ensemble on task flores_small2...
Start serving model...
Check model loading status...
HTTP connection has not yet been set up. Sleep 30...
2021-08-13 14:27:16,121 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Initializing plugins manager...
2021-08-13 14:27:16,552 [INFO ] main org.pytorch.serve.ModelServer -
Torchserve version: 0.4.2
TS Home: /usr/local/lib/python3.6/dist-packages
Current directory: /home/model-server/code
Temp directory: /home/model-server/tmp
Number of GPUs: 0
Number of CPUs: 16
Max heap size: 30688 M
Python executable: /usr/bin/python
Config file: N/A
Inference address: http://127.0.0.1:8080
Management address: http://127.0.0.1:8081
Metrics address: http://127.0.0.1:8082
Model Store: /opt/ml/model
Initial Models: adapter-ensemble.mar
Log dir: /home/model-server/code/logs
Metrics dir: /home/model-server/code/logs
Netty threads: 0
Netty client threads: 0
Default workers per model: 16
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Prefer direct buffer: false
Allowed Urls: [file://.*|http(s)?://.*]
Custom python dependency for model allowed: false
Metrics report format: prometheus
Enable metrics API: true
Workflow Store: /opt/ml/model
Model config: N/A
2021-08-13 14:27:16,563 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Loading snapshot serializer plugin...
2021-08-13 14:27:16,590 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: adapter-ensemble.mar
HTTP connection has not yet been set up. Sleep 30...
The last line then duplicates over and over the longer I let it run, suggesting that this is the reason the code is hanging. A quick search for this bug didn't turn up anything useful. The model already passed the local test.
Thank you in advance for your help.
Hello,
I've previously been able to get models to pass tests, but tried again a few hours ago with a new model and am getting a weird (lack of) result from
dynalab-cli test: the last few lines of output look like this:It's been hanging like this for a couple of hours. I tried running
docker run adapter-ensembleand got this:The last line then duplicates over and over the longer I let it run, suggesting that this is the reason the code is hanging. A quick search for this bug didn't turn up anything useful. The model already passed the local test.
Thank you in advance for your help.