As suggested by @phkrief , the idea of this issue is to share the experience when analysing projects with CROSSMINER, and trigger discussions to identify possible bugs/limitations on the platform. This issue is related to:
The default docker-compose may drain out the resources of a machine (even a powerful one). Thus, in order to get projects analysed, a solution (more details here) consists of:
- limiting the services in the docker compose. A reduced version of the docker compose is available here
- limiting the metrics provider selected when defining a task analysis
- limiting the time interval when defining a task analysis
Nevertheless, in my specific case, sometimes oss-app freezed (probably due to the limitation of my machine, something @MarcioMateus agreed on) and I had to delete the oss-db container. Furthermore, also queuing new task analysis was causing the current task to stop. Thus, I was waiting for a task to finish before adding a new task. Then, I was importing the data to elasticsearch with the script available at: https://github.com/valeriocos/scava/blob/bit/web-dashboards/scava-metrics/scava2es_battery.py (which calls scava2es on a battery of repos).
With what commented above, I was able to analyze all CHAOSS repos plus puppet-elasticsearch from 01/01/2019 to 30/06/2019 using the following metrics providers:
sentiment.SentimentHistoricMetricProvider
severity.SeverityHistoricMetricProvider
severitybugstatus.SeverityBugStatusHistoricMetricProvider
severityresponsetime.SeverityResponseTimeHistoricMetricProvider
severitysentiment.SeveritySentimentHistoricMetricProvider
newbugs.NewBugsHistoricMetricProvider
comments.CommentsHistoricMetricProvider
patches.PatchesHistoricMetricProvider
emotions.EmotionsHistoricMetricProvider
docker.dependencies
docker.smells
topics.TopicsHistoricMetricProvider
As suggested by @creat89 here, I had a quick look to the metrics. I noticed that the ones related to dev dependencies metric providers (e.g. osgi and maven) seem to eat up a considerable amount of memory. I checked it by selecting https://github.com/elastic/elasticsearch as target project (from 01/01/2019 to 30/06/2019), and looking at the memory consumption with slimbook@slimbook-KATANA:~$ top
As suggested by @phkrief , the idea of this issue is to share the experience when analysing projects with CROSSMINER, and trigger discussions to identify possible bugs/limitations on the platform. This issue is related to:
The default docker-compose may drain out the resources of a machine (even a powerful one). Thus, in order to get projects analysed, a solution (more details here) consists of:
Nevertheless, in my specific case, sometimes
oss-appfreezed (probably due to the limitation of my machine, something @MarcioMateus agreed on) and I had to delete theoss-dbcontainer. Furthermore, also queuing new task analysis was causing the current task to stop. Thus, I was waiting for a task to finish before adding a new task. Then, I was importing the data to elasticsearch with the script available at: https://github.com/valeriocos/scava/blob/bit/web-dashboards/scava-metrics/scava2es_battery.py (which calls scava2es on a battery of repos).With what commented above, I was able to analyze all CHAOSS repos plus puppet-elasticsearch from 01/01/2019 to 30/06/2019 using the following metrics providers:
As suggested by @creat89 here, I had a quick look to the metrics. I noticed that the ones related to dev dependencies metric providers (e.g. osgi and maven) seem to eat up a considerable amount of memory. I checked it by selecting https://github.com/elastic/elasticsearch as target project (from 01/01/2019 to 30/06/2019), and looking at the memory consumption with
slimbook@slimbook-KATANA:~$ top