Better handling of constants in -profile-ir

[AlexandreEichenberger]

Right now, when we do profiling of the IR, we get some constants (some are included, some are removed).

With this PR, the default with -profile-ir is to have as few constants in the runtime log as possible.

But if the new option -profile-all-ops is set (default off), then the profiling attempts to profile all constants, as well as the ops that may have been removed (by optimizations).

This allow us to get an idea of the memory pressure due to constants, for example by compiling with -profile-ir=ZHigh -profile-all-ops and log the output to a file.

Then this command will show all of the constants in the model

 make-report.py -r run2.log -s sig -f Const -l 2
Analyse SIG
Gather stats from 1 measurement sets with 0 warmup; keep inner 1 experiment(s)
Report using runtime file "run2.log" and compile file "run2.log"
Statistic legend:
   num, average time (s), cumulative time (s), percent of total , comma separated list of shapes with inputs followed by results


Statistics start all ops ordered_by time, tot_time,  0.0005820
  onnx.Constant, 968, 0.0000004, 0.0003580, 61.5%
    385, 0.0000004, 0.0001411, 39.4%, sig, 3xint64_t
    194, 0.0000004, 0.0000740, 20.7%, sig, 1600xfloat
    192, 0.0000004, 0.0000704, 19.7%, sig, 4xint64_t
    144, 0.0000004, 0.0000528, 14.7%, sig, 2xint64_t
    48, 0.0000004, 0.0000176, 4.9%, sig, 1x1x6x6x_Bool
    1, 0.0000010, 0.0000010, 0.3%, sig, 50257x1600xfloat
    2, 0.0000004, 0.0000007, 0.2%, sig, 1xint64_t
    1, 0.0000004, 0.0000004, 0.1%, sig, 1x6x1600xfloat
    1, 0.0000000, 0.0000000, 0.0%, sig, float
  zhigh.StickifiedConstant, 436, 0.0000005, 0.0002240, 38.5%
    97, 0.0000005, 0.0000485, 21.7%, sig, 1x25x1x1x32x64x_Float16
    48, 0.0000006, 0.0000290, 12.9%, sig, 1x1x25x1x32x64x_Float16
    48, 0.0000006, 0.0000265, 11.8%, sig, 1x100x1x50x32x64x_Float16
    48, 0.0000006, 0.0000265, 11.8%, sig, 1x100x1x1x32x64x_Float16
    48, 0.0000005, 0.0000255, 11.4%, sig, 1x25x1x200x32x64x_Float16
    48, 0.0000005, 0.0000225, 10.0%, sig, 1x75x1x50x32x64x_Float16
    48, 0.0000005, 0.0000225, 10.0%, sig, 1x75x1x1x32x64x_Float16
    48, 0.0000005, 0.0000220, 9.8%, sig, 1x25x1x50x32x64x_Float16
    1, 0.0000010, 0.0000010, 0.4%, sig, 1x786x1x50x32x64x_Float16
    2, 0.0000000, 0.0000000, 0.0%, sig, 1x1x1x1x32x64x_Float16
Statistics end all ops ordered_by time, tot_time,  0.0005820

where it lists all of the different shapes, and indicates how many of each of the shapes are present.

[chentong319]

Can we unify "profileAllOp" and "SkipConstant" into one name? If more op may be affected by this option in future, keep profileAllOp. Otherwise use SkipProfileConstant.
Also follow our convention to use "enable" or "disable" as prefix.

[AlexandreEichenberger]

Can we unify "profileAllOp" and "SkipConstant" into one name?

That is a good point. At this time, when we "profileAllOps", I explicitly add profiling for the constant AND disable removing of empty profiling (namely where was an operation that was profiled, but that operation vanished).

When we do not "prodileAllOps", then I don't add profiling for the constant AND remove empty profiling.

That is why I have 2 names. But maybe when I don't have the constant, there are not really any empty operations... I can check and let you know.

thanks @chentong319 .

[AlexandreEichenberger]

Implemented the requested changes, now the option is --enable-constant-op-profiling... a bit long but more precise. Thanks for the feedback.

[chentong319]

LGTM!

[tungld]

LGTM!

[tungld]

Just curious about why we need to add krnl.global here while this is for profiling onnx ops. I though there were no krnl.global ops at this IR level.

[AlexandreEichenberger]

Let me double check