Snakemake Workflow On Split Files

Perhaps this question has already been answered but I could not come up with the correct query to find it... I have a big file that needs to be analyzed. In order to do this quickl

Solution 1:

The rule "process_small_files" sees that it needs a file such as bigfile.001, but as far as snakemake knows, no rule in the workflow can make that file. While split_big_file will make that file, in the "output" section it only states that it will make the file "splitting_file.done" so snakemake doesn't think the workflow can make bigfile.001 and assumes it should already exist.

Because the split command makes a different number of files depending on the size of the input file, you will need to use the dynamic files feature of snakemake: https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#dynamic-files