Skip to content

Unused files in process inputs are of nextflow.processor.TaskPath type which makes invalid input paths for other processes #7032

@nvnieuwk

Description

@nvnieuwk

Bug report

Expected behavior and actual behavior

Files in records that are not part of a defined record type in a process input are converted to nextflow.processor.TaskPath. This class is an implementation of the Path type but returns the relative path to the file in the work directory of the first process, which is invalid in the second process. This causes an error when trying to run another process on the output of the first process (see below). I fixed this by running .toRealPath() on all TaskPath files but this adds a lot of redundant code. It would help a lot if either all output files are properly converted back to a Path types using .toRealPath() or if TaskPath could be made valid for process inputs.

Steps to reproduce the problem

  1. Put the following code in a main.nf file:
nextflow.preview.types = true
workflow {
    def p1 = P1(channel.of(record(id: 'test', file: file('test.txt'))))
    P2(p1)
}

process P1 {
    input:
    input: Input

    output:
    input + record(file2: file('test2.txt'))

    script:
    """
    echo "test" > test2.txt
    """
}

process P2 {
    input:
    input: Input

    script:
    """
    cat ${input.file} ${input.file2}
    """
}

record Input {
    id: String
    file: Path
    file2: Path?
}
  1. Create a test.txt file: echo 'test' > test.txt
  2. Run Nextflow with the latest edge version: NXF_VER=26.03.2-edge nextflow run main.nf

Program output

$ NXF_VER=26.03.2-edge nextflow run main.nf 

 N E X T F L O W   ~  version 26.03.2-edge

Launching `main.nf` [hopeful_wilson] revision: 5b69a8f056

WARN: Static typing is a preview feature -- syntax and behavior may change in future releases
executor >  local (1)
[8d/8f5dd7] process > P1 (1) [100%] 1 of 1 ✔
[-        ] process > P2     -
WARN: File system is unable to get file attributes file: test.txt -- Cause: java.nio.file.ProviderMismatchException
ERROR ~ Error executing process > 'P2 (1)'

Caused by:
  Something went wrong while creating task hash for process 'P2' -- Offending keys: [
   - type=java.util.UUID value=2439a953-c814-43dd-b9c1-358bdb5d8b48, 
   - type=java.lang.String value=P2, 
   - type=java.lang.String value=    """
      cat ${input.file} ${input.file2}
      """
  , 
   - type=java.lang.String value=input, 
   - type=nextflow.util.RecordMap value=[id:test, file:test.txt, file2:/home/nvnieuwk/Documents/nextflow/testing/taskpath/work/8d/8f5dd7b0203d3e12239bc894190607/test2.txt]]



 -- Check '.nextflow.log' file for details

Environment

  • Nextflow version: 26.03.2-edge
  • Java version: 17.0.3
  • Operating system: Linux
  • Bash version: 3.0.17

Additional context

(Add any other context about the problem here)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions