Whitelist filtering

[bamueh]

I was under the impression that passing a whitelist to filter-fasta.py would remove all sequences except those that are in the whitelist. Instead, it seems like nothing gets filtered at all. But maybe I'm misunderstanding something?

[terrycojones]

How did you pass a whitelist? There are two options, --whitelist and --whitelistFile. If you look in test/test_filter.py you'll see quite a few whitelist tests. I'm guessing you did --whitelist FILE instead of --whitelistFile FILE ? (We could easily make a change to detect this and give a warning.)

[bamueh]

No, I did --whitelist somesequencename and --whitelistFile FILE and both didn't work.

[bamueh]

e.g.

$ filter-fasta.py --whitelist OP866294_C_C2.2_Ethiopia_Camelus_dromedarius_2019-11-23 < data/sequences.fasta > sss.fasta
Read 640 sequences, kept 640 (100.00%).

where OP866294_C_C2.2_Ethiopia_Camelus_dromedarius_2019-11-23 is a name of a sequence in data/sequences.fasta.

I would have expected that only one sequence would be kept (OP866294_C_C2.2_Ethiopia_Camelus_dromedarius_2019-11-23) and not 640, but maybe I'm misunderstanding something?

[terrycojones]

OK, got it. A whitelist is used to guarantee that some things don't get thrown away. A blacklist ensures that some things do. The whitelist has priority. So what you should do is:

$ filter-fasta.py --whitelist OP866294_C_C2.2_Ethiopia_Camelus_dromedarius_2019-11-23 \
    --negativeTitleRegex . < data/sequences.fasta > sss.fasta

I.e., you ask that everything be thrown away, except the ones on the whitelist.

[terrycojones]

BTW, these terms are now considered racist.

[terrycojones]

If you look at the tests, you'll see several that have negativeRegex="."