Flagging and filtering out bad quality IFCB samples

[mdugenne]

Known issues (leak, air bubbles, camera focus, and settings) with the IFCB instrument may alter the detection of plankton and result in inaccurate count and/or size estimates, affecting the observed size distribution and predicted slope of the size spectrum.

Given that (1) IFCB projects uploaded on Ecotaxa have not been flagged, (2) only a small percentage of IFCB projects have been validated (3) some predicted artefacts are actually good quality images of plankton, is there a reliable way to flag and filter out bad quality IFCB samples?

Few ideas: Within the possible artefacts (e.g. bubbles, beads, badfocus), bubbles are likely the best predicted, followed by beads.

[MarCorralesU]

considering that we don't want to get rid of data (rows) for the "cleaned" version of the tsv files, this part of the file reading function

PSSdb/scripts/Size_spectra_functions.py

Lines 75 to 82 in cd3116e

	# FILTERING DATA: filter taxonomic categories that are artefacts
	data_clean = df[df['object_annotation_category'].str.contains("artefact") == False]
	# remove data points where size metrics = 0
	data_clean = data_clean[data_clean.object_equivdiameter != 0]
	# finally, remove any row with coordinates =nan
	data_clean = data_clean[data_clean['object_lat'].notna()]
	data_clean = data_clean[data_clean['object_lon'].notna()]
	data_clean = data_clean.reset_index()

needs to be included after the data is read and standardized but before the data is used for PSS analysis