Compare E2 and C2 nodes

[katilp]

Price difference:
e2-standard-4 with 100Gb disk: $521.43 / month
c2-standard-4 with 100Gb disk: $761.83 / month

e2/c2 = 68%
c2/e2 = 1.46

(source price calculator: https://cloud.google.com/products/calculator?hl=en)

Test job (using a secondary boot disk with the pfnano image, i.e. without the image pull overhead)
10000 events single job on a 2-node cluster:

e2-standard-4

$ argo get -n argo @latest
Name:                pfnano-process-m8fqf
Namespace:           argo
ServiceAccount:      unset (will run with the default ServiceAccount)
Status:              Succeeded
Conditions:
 PodRunning          False
 Completed           True
Created:             Sat Sep 21 22:35:30 +0200 (29 minutes ago)
Started:             Sat Sep 21 22:35:30 +0200 (29 minutes ago)
Finished:            Sat Sep 21 23:04:27 +0200 (39 seconds ago)
Duration:            28 minutes 57 seconds
Progress:            4/4
ResourcesDuration:   22m5s*(1 cpu),13m36s*(10Gi ephemeral-storage),7h16m41s*(100Mi memory)
Parameters:
  startFile:         1
  nEvents:           10000
  recid:             30541
  nJobs:             1
  bucket:            test-gcs-argo-bucket

STEP                                                                TEMPLATE               PODNAME                                                DURATION  MESSAGE
 ✔ pfnano-process-m8fqf                                             cms-od-example
 ├─✔ get-metadata                                                   get-metadata-template  pfnano-process-m8fqf-get-metadata-template-3548506966  9s
 ├─✔ joblist                                                        joblist-template       pfnano-process-m8fqf-joblist-template-1058542947       6s
 ├─✔ runpfnano(0:eventsinjob:10000,firstfile:1,it:1,lastfile:1309)  runpfnano-template     pfnano-process-m8fqf-runpfnano-template-405427176      27m
 └─✔ plot                                                           plot-template          pfnano-process-m8fqf-plot-template-1849748679          57s

c2-standard-4

$ argo get -n argo @latest
Name:                pfnano-process-wc5dt
Namespace:           argo
ServiceAccount:      unset (will run with the default ServiceAccount)
Status:              Succeeded
Conditions:
 PodRunning          False
 Completed           True
Created:             Sat Sep 21 23:34:50 +0200 (21 minutes ago)
Started:             Sat Sep 21 23:34:50 +0200 (21 minutes ago)
Finished:            Sat Sep 21 23:56:10 +0200 (3 seconds ago)
Duration:            21 minutes 20 seconds
Progress:            4/4
ResourcesDuration:   15m42s*(1 cpu),9m40s*(10Gi ephemeral-storage),5h10m54s*(100Mi memory)
Parameters:
  startFile:         1
  nEvents:           10000
  recid:             30541
  nJobs:             1
  bucket:            test-gcs-argo-bucket

STEP                                                                TEMPLATE               PODNAME                                                DURATION  MESSAGE
 ✔ pfnano-process-wc5dt                                             cms-od-example
 ├─✔ get-metadata                                                   get-metadata-template  pfnano-process-wc5dt-get-metadata-template-3955631283  18s
 ├─✔ joblist                                                        joblist-template       pfnano-process-wc5dt-joblist-template-4190849420       6s
 ├─✔ runpfnano(0:eventsinjob:10000,firstfile:1,it:1,lastfile:1309)  runpfnano-template     pfnano-process-wc5dt-runpfnano-template-1833562195     19m
 └─✔ plot                                                           plot-template          pfnano-process-wc5dt-plot-template-1413463354          50s

NB somewhat higher memory consumption

$ kubectl top pod -n argo
NAME                                                 CPU(cores)   MEMORY(bytes)

pfnano-process-wc5dt-runpfnano-template-1833562195   1000m        1887Mi

Timing

c2: 73 % of e2
e2: 1.36 x c2

[katilp]

The memory usage is also higher in the e2-standard-4 machine. Maybe due to image streaming enabled for the cluster (needed to use the secondary boot disk)?