Ingest New Sources from Sanghi23#588
Conversation
canavarrete01
changed the title
|
Added a new script to ingest all alternative names from the original Ultracool sheet |
canavarrete01
requested review from
Copilot and
kelle
and removed request for
Copilot
There was a problem hiding this comment.
Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.
Files not reviewed (1)
- scripts/ingests/sanghi23/NewSources-23.csv: Language not supported
Comments suppressed due to low confidence (1)
scripts/ingests/sanghi23/IngestAllNames.py:39
- Using the string 'Null' for null checks might be unreliable; consider using a standard null check (e.g., against None) or verifying that the string is non-empty.
if name != db_source[0] and name != "Null":
| print(f"Total sources add: {n_added}/43") | ||
| print(f"Total sources skipped: {n_skipped}/43") | ||
|
|
There was a problem hiding this comment.
Consider changing 'Total sources add:' to 'Total sources added:' for correct grammar.
| print(f"Total sources add: {n_added}/43") | |
| print(f"Total sources skipped: {n_skipped}/43") | |
| print(f"Total sources added: {n_added}/43") |
| # # Ingest Alternative Sources --- | ||
| print(f"Ingesting alternative names for {n_added} sources.") | ||
| for _, source in newsources.iterrows(): | ||
| if source['name_simbadable'] != "Null": |
There was a problem hiding this comment.
Using the string literal 'Null' for null checks may lead to errors if the data representation changes. Consider checking against None or an empty string for robustness.
| if source['name_simbadable'] != "Null": | |
| if source['name_simbadable'] is not None and source['name_simbadable'] != "": |
kelle
left a comment
kelle
left a comment
There was a problem hiding this comment.
I opened a PR to your branch to add some counters. As I said there, if your numbers agree with what I got -- 'Added 243 names to the database, 559 names were already present.' -- then go ahead and commit the 243 JSON files. Let's get the alt names merged and then tackle the new sources in their own PR.
|
These have all been resolved now! @kelle |
|
I don't see the JSON files. |
* Ingesting Sources * Minor updates to Ingest script * Ingest Sources and Sources' Alt Names * add counters * Updates to tests * Adding json files * Updating YU2 Sources * Deleted old YU files * Update test * version update --------- Co-authored-by: kelle <kellecruz@gmail.com>
Short description: Include what type of data being ingested and appropriate references.
Link to relevant issue: Closes #
For data ingests:
[x ] includes script used for ingest
[ ] includes modified JSON files
[ ] Add new tests
[ ] Update the Versions table