HPCG-NVIDIA 24.09.0 -- NVIDIA accelerated HPCG benchmark -- NVIDIA Build v0.5.6 Start of application (GPU-Only) ... Initial Residual = 103467 Iteration = 1 Scaled Residual = 0.189374 Iteration = 2 Scaled Residual = 0.10447 Iteration = 3 Scaled Residual = 0.0722339 Iteration = 4 Scaled Residual = 0.0552294 Iteration = 5 Scaled Residual = 0.044677 Iteration = 6 Scaled Residual = 0.0374925 Iteration = 7 Scaled Residual = 0.0323008 Iteration = 8 Scaled Residual = 0.0283949 Iteration = 9 Scaled Residual = 0.0253646 Iteration = 10 Scaled Residual = 0.0229465 Iteration = 11 Scaled Residual = 0.0209632 Iteration = 12 Scaled Residual = 0.0192923 Iteration = 13 Scaled Residual = 0.0178507 Iteration = 14 Scaled Residual = 0.0165891 Iteration = 15 Scaled Residual = 0.0154801 Iteration = 16 Scaled Residual = 0.0145069 Iteration = 17 Scaled Residual = 0.0136572 Iteration = 18 Scaled Residual = 0.0129174 Iteration = 19 Scaled Residual = 0.0122697 Iteration = 20 Scaled Residual = 0.0116952 Iteration = 21 Scaled Residual = 0.0111748 Iteration = 22 Scaled Residual = 0.0106933 Iteration = 23 Scaled Residual = 0.0102439 Iteration = 24 Scaled Residual = 0.00982711 Iteration = 25 Scaled Residual = 0.00944695 Iteration = 26 Scaled Residual = 0.00910876 Iteration = 27 Scaled Residual = 0.00881422 Iteration = 28 Scaled Residual = 0.00855898 Iteration = 29 Scaled Residual = 0.00833377 Iteration = 30 Scaled Residual = 0.00812507 Iteration = 31 Scaled Residual = 0.00792012 Iteration = 32 Scaled Residual = 0.00771309 Iteration = 33 Scaled Residual = 0.00750484 Iteration = 34 Scaled Residual = 0.00730025 Iteration = 35 Scaled Residual = 0.00710366 Iteration = 36 Scaled Residual = 0.00691379 Iteration = 37 Scaled Residual = 0.00672526 Iteration = 38 Scaled Residual = 0.00653347 Iteration = 39 Scaled Residual = 0.00633832 Iteration = 40 Scaled Residual = 0.00614664 Iteration = 41 Scaled Residual = 0.0059688 Iteration = 42 Scaled Residual = 0.00581282 Iteration = 43 Scaled Residual = 0.00568236 Iteration = 44 Scaled Residual = 0.00557543 Iteration = 45 Scaled Residual = 0.0054838 Iteration = 46 Scaled Residual = 0.00539697 Iteration = 47 Scaled Residual = 0.00530669 Iteration = 48 Scaled Residual = 0.00520939 Iteration = 49 Scaled Residual = 0.00510778 Iteration = 50 Scaled Residual = 0.00500715 GPU Rank Info: | cuSPARSE version 12.5 | Reference CPU memory = 29957.77 MB | GPU Name: 'NVIDIA H100 80GB HBM3' | GPU Memory Use: 54265 MB / 81109 MB | Process Grid: 10x12x12 | Local Domain: 512x512x256 | Number of CPU Threads: 14 | Slice Size: 2048 WARNING: PERFORMING UNPRECONDITIONED ITERATIONS Call [0] Number of Iterations [11] Scaled Residual [1.19807e-14] WARNING: PERFORMING UNPRECONDITIONED ITERATIONS Call [1] Number of Iterations [11] Scaled Residual [1.19807e-14] Call [0] Number of Iterations [2] Scaled Residual [1.55631e-17] Call [1] Number of Iterations [2] Scaled Residual [1.55631e-17] Departure from symmetry (scaled) for SpMV abs(x'*A*y - y'*A*x) = 0 Departure from symmetry (scaled) for MG abs(x'*Minv*y - y'*Minv*x) = 0 SpMV call [0] Residual [0] SpMV call [1] Residual [0] Initial Residual = 103467 Iteration = 1 Scaled Residual = 0.220471 Iteration = 2 Scaled Residual = 0.11929 Iteration = 3 Scaled Residual = 0.0812262 Iteration = 4 Scaled Residual = 0.0615748 Iteration = 5 Scaled Residual = 0.0496445 Iteration = 6 Scaled Residual = 0.0415901 Iteration = 7 Scaled Residual = 0.0358016 Iteration = 8 Scaled Residual = 0.0314457 Iteration = 9 Scaled Residual = 0.0280369 Iteration = 10 Scaled Residual = 0.025301 Iteration = 11 Scaled Residual = 0.0230572 Iteration = 12 Scaled Residual = 0.0211799 Iteration = 13 Scaled Residual = 0.0195911 Iteration = 14 Scaled Residual = 0.0182278 Iteration = 15 Scaled Residual = 0.0170436 Iteration = 16 Scaled Residual = 0.0160078 Iteration = 17 Scaled Residual = 0.0150945 Iteration = 18 Scaled Residual = 0.0142846 Iteration = 19 Scaled Residual = 0.0135627 Iteration = 20 Scaled Residual = 0.0129155 Iteration = 21 Scaled Residual = 0.0123338 Iteration = 22 Scaled Residual = 0.0118093 Iteration = 23 Scaled Residual = 0.0113347 Iteration = 24 Scaled Residual = 0.0109035 Iteration = 25 Scaled Residual = 0.0105083 Iteration = 26 Scaled Residual = 0.0101439 Iteration = 27 Scaled Residual = 0.00980472 Iteration = 28 Scaled Residual = 0.00948496 Iteration = 29 Scaled Residual = 0.00918062 Iteration = 30 Scaled Residual = 0.0088881 Iteration = 31 Scaled Residual = 0.00860557 Iteration = 32 Scaled Residual = 0.00833367 Iteration = 33 Scaled Residual = 0.0080725 Iteration = 34 Scaled Residual = 0.00782338 Iteration = 35 Scaled Residual = 0.00758823 Iteration = 36 Scaled Residual = 0.0073676 Iteration = 37 Scaled Residual = 0.00716201 Iteration = 38 Scaled Residual = 0.00697151 Iteration = 39 Scaled Residual = 0.0067948 Iteration = 40 Scaled Residual = 0.00663128 Iteration = 41 Scaled Residual = 0.00647994 Iteration = 42 Scaled Residual = 0.00633866 Iteration = 43 Scaled Residual = 0.00620564 Iteration = 44 Scaled Residual = 0.00607884 Iteration = 45 Scaled Residual = 0.00595582 Iteration = 46 Scaled Residual = 0.00583507 Iteration = 47 Scaled Residual = 0.0057148 Iteration = 48 Scaled Residual = 0.00559399 Iteration = 49 Scaled Residual = 0.00547369 Iteration = 50 Scaled Residual = 0.00535504 Iteration = 51 Scaled Residual = 0.00523956 Iteration = 52 Scaled Residual = 0.00512962 Iteration = 53 Scaled Residual = 0.00502643 Iteration = 54 Scaled Residual = 0.00493097 Call [0] Scaled Residual [0.00493097] Call [1] Scaled Residual [0.00493097] Call [2] Scaled Residual [0.00493097] Call [3] Scaled Residual [0.00493097] Call [4] Scaled Residual [0.00493097] Call [5] Scaled Residual [0.00493097] Call [6] Scaled Residual [0.00493097] Call [7] Scaled Residual [0.00493097] Call [8] Scaled Residual [0.00493097] Call [9] Scaled Residual [0.00493097] Call [10] Scaled Residual [0.00493097] Call [11] Scaled Residual [0.00493097] Call [12] Scaled Residual [0.00493097] Call [13] Scaled Residual [0.00493097] Call [14] Scaled Residual [0.00493097] Call [15] Scaled Residual [0.00493097] Call [16] Scaled Residual [0.00493097] Call [17] Scaled Residual [0.00493097] Call [18] Scaled Residual [0.00493097] Call [19] Scaled Residual [0.00493097] Call [20] Scaled Residual [0.00493097] Call [21] Scaled Residual [0.00493097] Call [22] Scaled Residual [0.00493097] Call [23] Scaled Residual [0.00493097] Call [24] Scaled Residual [0.00493097] Call [25] Scaled Residual [0.00493097] Call [26] Scaled Residual [0.00493097] Call [27] Scaled Residual [0.00493097] Call [28] Scaled Residual [0.00493097] Call [29] Scaled Residual [0.00493097] Call [30] Scaled Residual [0.00493097] Call [31] Scaled Residual [0.00493097] Call [32] Scaled Residual [0.00493097] Call [33] Scaled Residual [0.00493097] Call [34] Scaled Residual [0.00493097] Call [35] Scaled Residual [0.00493097] Call [36] Scaled Residual [0.00493097] Call [37] Scaled Residual [0.00493097] Call [38] Scaled Residual [0.00493097] Call [39] Scaled Residual [0.00493097] Call [40] Scaled Residual [0.00493097] Call [41] Scaled Residual [0.00493097] Call [42] Scaled Residual [0.00493097] Call [43] Scaled Residual [0.00493097] Call [44] Scaled Residual [0.00493097] Call [45] Scaled Residual [0.00493097] Call [46] Scaled Residual [0.00493097] Call [47] Scaled Residual [0.00493097] Call [48] Scaled Residual [0.00493097] Call [49] Scaled Residual [0.00493097] Call [50] Scaled Residual [0.00493097] Call [51] Scaled Residual [0.00493097] Call [52] Scaled Residual [0.00493097] Call [53] Scaled Residual [0.00493097] Call [54] Scaled Residual [0.00493097] Call [55] Scaled Residual [0.00493097] Call [56] Scaled Residual [0.00493097] Call [57] Scaled Residual [0.00493097] Call [58] Scaled Residual [0.00493097] Call [59] Scaled Residual [0.00493097] Call [60] Scaled Residual [0.00493097] Call [61] Scaled Residual [0.00493097] Call [62] Scaled Residual [0.00493097] Call [63] Scaled Residual [0.00493097] Call [64] Scaled Residual [0.00493097] Call [65] Scaled Residual [0.00493097] Call [66] Scaled Residual [0.00493097] Call [67] Scaled Residual [0.00493097] Call [68] Scaled Residual [0.00493097] Call [69] Scaled Residual [0.00493097] Call [70] Scaled Residual [0.00493097] Call [71] Scaled Residual [0.00493097] Call [72] Scaled Residual [0.00493097] Call [73] Scaled Residual [0.00493097] Call [74] Scaled Residual [0.00493097] Call [75] Scaled Residual [0.00493097] Call [76] Scaled Residual [0.00493097] Call [77] Scaled Residual [0.00493097] Call [78] Scaled Residual [0.00493097] Call [79] Scaled Residual [0.00493097] Call [80] Scaled Residual [0.00493097] Call [81] Scaled Residual [0.00493097] Call [82] Scaled Residual [0.00493097] Call [83] Scaled Residual [0.00493097] Call [84] Scaled Residual [0.00493097] Call [85] Scaled Residual [0.00493097] Call [86] Scaled Residual [0.00493097] Call [87] Scaled Residual [0.00493097] Call [88] Scaled Residual [0.00493097] Call [89] Scaled Residual [0.00493097] Call [90] Scaled Residual [0.00493097] Call [91] Scaled Residual [0.00493097] Call [92] Scaled Residual [0.00493097] Call [93] Scaled Residual [0.00493097] Call [94] Scaled Residual [0.00493097] Call [95] Scaled Residual [0.00493097] Call [96] Scaled Residual [0.00493097] Call [97] Scaled Residual [0.00493097] Call [98] Scaled Residual [0.00493097] Call [99] Scaled Residual [0.00493097] Call [100] Scaled Residual [0.00493097] Call [101] Scaled Residual [0.00493097] Call [102] Scaled Residual [0.00493097] Call [103] Scaled Residual [0.00493097] Call [104] Scaled Residual [0.00493097] Call [105] Scaled Residual [0.00493097] Call [106] Scaled Residual [0.00493097] Call [107] Scaled Residual [0.00493097] Call [108] Scaled Residual [0.00493097] Call [109] Scaled Residual [0.00493097] Call [110] Scaled Residual [0.00493097] Call [111] Scaled Residual [0.00493097] Call [112] Scaled Residual [0.00493097] Call [113] Scaled Residual [0.00493097] Call [114] Scaled Residual [0.00493097] Call [115] Scaled Residual [0.00493097] Call [116] Scaled Residual [0.00493097] Call [117] Scaled Residual [0.00493097] Call [118] Scaled Residual [0.00493097] Call [119] Scaled Residual [0.00493097] Call [120] Scaled Residual [0.00493097] Call [121] Scaled Residual [0.00493097] Call [122] Scaled Residual [0.00493097] Call [123] Scaled Residual [0.00493097] Call [124] Scaled Residual [0.00493097] Call [125] Scaled Residual [0.00493097] Call [126] Scaled Residual [0.00493097] Call [127] Scaled Residual [0.00493097] Call [128] Scaled Residual [0.00493097] Call [129] Scaled Residual [0.00493097] Call [130] Scaled Residual [0.00493097] Call [131] Scaled Residual [0.00493097] Call [132] Scaled Residual [0.00493097] Call [133] Scaled Residual [0.00493097] Call [134] Scaled Residual [0.00493097] Call [135] Scaled Residual [0.00493097] Call [136] Scaled Residual [0.00493097] Call [137] Scaled Residual [0.00493097] Call [138] Scaled Residual [0.00493097] Call [139] Scaled Residual [0.00493097] Call [140] Scaled Residual [0.00493097] Call [141] Scaled Residual [0.00493097] Call [142] Scaled Residual [0.00493097] Call [143] Scaled Residual [0.00493097] Call [144] Scaled Residual [0.00493097] Call [145] Scaled Residual [0.00493097] Call [146] Scaled Residual [0.00493097] Call [147] Scaled Residual [0.00493097] Call [148] Scaled Residual [0.00493097] Call [149] Scaled Residual [0.00493097] Call [150] Scaled Residual [0.00493097] Call [151] Scaled Residual [0.00493097] Call [152] Scaled Residual [0.00493097] Call [153] Scaled Residual [0.00493097] Call [154] Scaled Residual [0.00493097] Call [155] Scaled Residual [0.00493097] Call [156] Scaled Residual [0.00493097] Call [157] Scaled Residual [0.00493097] Call [158] Scaled Residual [0.00493097] Call [159] Scaled Residual [0.00493097] Call [160] Scaled Residual [0.00493097] Call [161] Scaled Residual [0.00493097] Call [162] Scaled Residual [0.00493097] Call [163] Scaled Residual [0.00493097] Call [164] Scaled Residual [0.00493097] Call [165] Scaled Residual [0.00493097] Call [166] Scaled Residual [0.00493097] Call [167] Scaled Residual [0.00493097] Call [168] Scaled Residual [0.00493097] Call [169] Scaled Residual [0.00493097] Call [170] Scaled Residual [0.00493097] Call [171] Scaled Residual [0.00493097] Call [172] Scaled Residual [0.00493097] Call [173] Scaled Residual [0.00493097] Call [174] Scaled Residual [0.00493097] Call [175] Scaled Residual [0.00493097] Call [176] Scaled Residual [0.00493097] Call [177] Scaled Residual [0.00493097] Call [178] Scaled Residual [0.00493097] Call [179] Scaled Residual [0.00493097] Call [180] Scaled Residual [0.00493097] Call [181] Scaled Residual [0.00493097] Call [182] Scaled Residual [0.00493097] Call [183] Scaled Residual [0.00493097] Call [184] Scaled Residual [0.00493097] Call [185] Scaled Residual [0.00493097] Call [186] Scaled Residual [0.00493097] Call [187] Scaled Residual [0.00493097] Call [188] Scaled Residual [0.00493097] Call [189] Scaled Residual [0.00493097] Call [190] Scaled Residual [0.00493097] Call [191] Scaled Residual [0.00493097] Call [192] Scaled Residual [0.00493097] Call [193] Scaled Residual [0.00493097] Call [194] Scaled Residual [0.00493097] Call [195] Scaled Residual [0.00493097] Call [196] Scaled Residual [0.00493097] Call [197] Scaled Residual [0.00493097] Call [198] Scaled Residual [0.00493097] Call [199] Scaled Residual [0.00493097] Call [200] Scaled Residual [0.00493097] Call [201] Scaled Residual [0.00493097] Call [202] Scaled Residual [0.00493097] Call [203] Scaled Residual [0.00493097] Call [204] Scaled Residual [0.00493097] Call [205] Scaled Residual [0.00493097] Call [206] Scaled Residual [0.00493097] Call [207] Scaled Residual [0.00493097] Call [208] Scaled Residual [0.00493097] Call [209] Scaled Residual [0.00493097] Call [210] Scaled Residual [0.00493097] Call [211] Scaled Residual [0.00493097] Call [212] Scaled Residual [0.00493097] Call [213] Scaled Residual [0.00493097] Call [214] Scaled Residual [0.00493097] Call [215] Scaled Residual [0.00493097] Call [216] Scaled Residual [0.00493097] Call [217] Scaled Residual [0.00493097] Call [218] Scaled Residual [0.00493097] Call [219] Scaled Residual [0.00493097] Call [220] Scaled Residual [0.00493097] Call [221] Scaled Residual [0.00493097] Call [222] Scaled Residual [0.00493097] Call [223] Scaled Residual [0.00493097] Call [224] Scaled Residual [0.00493097] Call [225] Scaled Residual [0.00493097] Call [226] Scaled Residual [0.00493097] Call [227] Scaled Residual [0.00493097] Call [228] Scaled Residual [0.00493097] Call [229] Scaled Residual [0.00493097] Call [230] Scaled Residual [0.00493097] Call [231] Scaled Residual [0.00493097] Call [232] Scaled Residual [0.00493097] Call [233] Scaled Residual [0.00493097] Call [234] Scaled Residual [0.00493097] Call [235] Scaled Residual [0.00493097] Call [236] Scaled Residual [0.00493097] Call [237] Scaled Residual [0.00493097] Call [238] Scaled Residual [0.00493097] Call [239] Scaled Residual [0.00493097] Call [240] Scaled Residual [0.00493097] Call [241] Scaled Residual [0.00493097] Call [242] Scaled Residual [0.00493097] Call [243] Scaled Residual [0.00493097] Call [244] Scaled Residual [0.00493097] Call [245] Scaled Residual [0.00493097] Call [246] Scaled Residual [0.00493097] Call [247] Scaled Residual [0.00493097] Call [248] Scaled Residual [0.00493097] Call [249] Scaled Residual [0.00493097] Call [250] Scaled Residual [0.00493097] Call [251] Scaled Residual [0.00493097] Call [252] Scaled Residual [0.00493097] Call [253] Scaled Residual [0.00493097] Call [254] Scaled Residual [0.00493097] Call [255] Scaled Residual [0.00493097] Call [256] Scaled Residual [0.00493097] Call [257] Scaled Residual [0.00493097] Call [258] Scaled Residual [0.00493097] Call [259] Scaled Residual [0.00493097] Call [260] Scaled Residual [0.00493097] Call [261] Scaled Residual [0.00493097] Call [262] Scaled Residual [0.00493097] Call [263] Scaled Residual [0.00493097] Call [264] Scaled Residual [0.00493097] Call [265] Scaled Residual [0.00493097] Call [266] Scaled Residual [0.00493097] Call [267] Scaled Residual [0.00493097] Call [268] Scaled Residual [0.00493097] Call [269] Scaled Residual [0.00493097] Call [270] Scaled Residual [0.00493097] Call [271] Scaled Residual [0.00493097] Call [272] Scaled Residual [0.00493097] Call [273] Scaled Residual [0.00493097] Call [274] Scaled Residual [0.00493097] Call [275] Scaled Residual [0.00493097] Call [276] Scaled Residual [0.00493097] Call [277] Scaled Residual [0.00493097] Call [278] Scaled Residual [0.00493097] Call [279] Scaled Residual [0.00493097] Call [280] Scaled Residual [0.00493097] Call [281] Scaled Residual [0.00493097] Call [282] Scaled Residual [0.00493097] Call [283] Scaled Residual [0.00493097] Call [284] Scaled Residual [0.00493097] Call [285] Scaled Residual [0.00493097] Call [286] Scaled Residual [0.00493097] Call [287] Scaled Residual [0.00493097] Call [288] Scaled Residual [0.00493097] Call [289] Scaled Residual [0.00493097] Call [290] Scaled Residual [0.00493097] Call [291] Scaled Residual [0.00493097] Call [292] Scaled Residual [0.00493097] Call [293] Scaled Residual [0.00493097] Call [294] Scaled Residual [0.00493097] Call [295] Scaled Residual [0.00493097] Call [296] Scaled Residual [0.00493097] Call [297] Scaled Residual [0.00493097] Call [298] Scaled Residual [0.00493097] Call [299] Scaled Residual [0.00493097] Call [300] Scaled Residual [0.00493097] Call [301] Scaled Residual [0.00493097] Call [302] Scaled Residual [0.00493097] Call [303] Scaled Residual [0.00493097] Call [304] Scaled Residual [0.00493097] Call [305] Scaled Residual [0.00493097] Call [306] Scaled Residual [0.00493097] Call [307] Scaled Residual [0.00493097] Call [308] Scaled Residual [0.00493097] Call [309] Scaled Residual [0.00493097] Call [310] Scaled Residual [0.00493097] Call [311] Scaled Residual [0.00493097] Call [312] Scaled Residual [0.00493097] Call [313] Scaled Residual [0.00493097] Call [314] Scaled Residual [0.00493097] Call [315] Scaled Residual [0.00493097] Call [316] Scaled Residual [0.00493097] Call [317] Scaled Residual [0.00493097] Call [318] Scaled Residual [0.00493097] Call [319] Scaled Residual [0.00493097] Call [320] Scaled Residual [0.00493097] Call [321] Scaled Residual [0.00493097] Call [322] Scaled Residual [0.00493097] Call [323] Scaled Residual [0.00493097] Call [324] Scaled Residual [0.00493097] Call [325] Scaled Residual [0.00493097] Call [326] Scaled Residual [0.00493097] Call [327] Scaled Residual [0.00493097] Call [328] Scaled Residual [0.00493097] Call [329] Scaled Residual [0.00493097] Call [330] Scaled Residual [0.00493097] Call [331] Scaled Residual [0.00493097] Call [332] Scaled Residual [0.00493097] Call [333] Scaled Residual [0.00493097] Call [334] Scaled Residual [0.00493097] Call [335] Scaled Residual [0.00493097] Call [336] Scaled Residual [0.00493097] Call [337] Scaled Residual [0.00493097] Call [338] Scaled Residual [0.00493097] Call [339] Scaled Residual [0.00493097] Call [340] Scaled Residual [0.00493097] Call [341] Scaled Residual [0.00493097] Call [342] Scaled Residual [0.00493097] Call [343] Scaled Residual [0.00493097] Call [344] Scaled Residual [0.00493097] Call [345] Scaled Residual [0.00493097] Call [346] Scaled Residual [0.00493097] Call [347] Scaled Residual [0.00493097] Call [348] Scaled Residual [0.00493097] Call [349] Scaled Residual [0.00493097] Call [350] Scaled Residual [0.00493097] Call [351] Scaled Residual [0.00493097] Call [352] Scaled Residual [0.00493097] Call [353] Scaled Residual [0.00493097] Call [354] Scaled Residual [0.00493097] Call [355] Scaled Residual [0.00493097] Call [356] Scaled Residual [0.00493097] Call [357] Scaled Residual [0.00493097] Call [358] Scaled Residual [0.00493097] Call [359] Scaled Residual [0.00493097] Call [360] Scaled Residual [0.00493097] Call [361] Scaled Residual [0.00493097] Call [362] Scaled Residual [0.00493097] Call [363] Scaled Residual [0.00493097] Call [364] Scaled Residual [0.00493097] Call [365] Scaled Residual [0.00493097] Call [366] Scaled Residual [0.00493097] Call [367] Scaled Residual [0.00493097] Call [368] Scaled Residual [0.00493097] Call [369] Scaled Residual [0.00493097] Call [370] Scaled Residual [0.00493097] Call [371] Scaled Residual [0.00493097] Call [372] Scaled Residual [0.00493097] Call [373] Scaled Residual [0.00493097] Call [374] Scaled Residual [0.00493097] Call [375] Scaled Residual [0.00493097] Call [376] Scaled Residual [0.00493097] Call [377] Scaled Residual [0.00493097] Call [378] Scaled Residual [0.00493097] Call [379] Scaled Residual [0.00493097] Call [380] Scaled Residual [0.00493097] Call [381] Scaled Residual [0.00493097] Call [382] Scaled Residual [0.00493097] Call [383] Scaled Residual [0.00493097] Call [384] Scaled Residual [0.00493097] Call [385] Scaled Residual [0.00493097] Call [386] Scaled Residual [0.00493097] Call [387] Scaled Residual [0.00493097] Call [388] Scaled Residual [0.00493097] Call [389] Scaled Residual [0.00493097] Call [390] Scaled Residual [0.00493097] Call [391] Scaled Residual [0.00493097] Call [392] Scaled Residual [0.00493097] Call [393] Scaled Residual [0.00493097] Call [394] Scaled Residual [0.00493097] Call [395] Scaled Residual [0.00493097] Call [396] Scaled Residual [0.00493097] Call [397] Scaled Residual [0.00493097] Call [398] Scaled Residual [0.00493097] Call [399] Scaled Residual [0.00493097] Call [400] Scaled Residual [0.00493097] Call [401] Scaled Residual [0.00493097] Call [402] Scaled Residual [0.00493097] Call [403] Scaled Residual [0.00493097] Call [404] Scaled Residual [0.00493097] Call [405] Scaled Residual [0.00493097] Call [406] Scaled Residual [0.00493097] Call [407] Scaled Residual [0.00493097] Call [408] Scaled Residual [0.00493097] Call [409] Scaled Residual [0.00493097] Call [410] Scaled Residual [0.00493097] Call [411] Scaled Residual [0.00493097] Call [412] Scaled Residual [0.00493097] Call [413] Scaled Residual [0.00493097] Call [414] Scaled Residual [0.00493097] Call [415] Scaled Residual [0.00493097] Call [416] Scaled Residual [0.00493097] Call [417] Scaled Residual [0.00493097] Call [418] Scaled Residual [0.00493097] Call [419] Scaled Residual [0.00493097] Call [420] Scaled Residual [0.00493097] Call [421] Scaled Residual [0.00493097] Call [422] Scaled Residual [0.00493097] Call [423] Scaled Residual [0.00493097] Call [424] Scaled Residual [0.00493097] Call [425] Scaled Residual [0.00493097] Call [426] Scaled Residual [0.00493097] Call [427] Scaled Residual [0.00493097] Call [428] Scaled Residual [0.00493097] Call [429] Scaled Residual [0.00493097] Call [430] Scaled Residual [0.00493097] Call [431] Scaled Residual [0.00493097] Call [432] Scaled Residual [0.00493097] Call [433] Scaled Residual [0.00493097] Call [434] Scaled Residual [0.00493097] Call [435] Scaled Residual [0.00493097] Call [436] Scaled Residual [0.00493097] Call [437] Scaled Residual [0.00493097] Call [438] Scaled Residual [0.00493097] Call [439] Scaled Residual [0.00493097] Call [440] Scaled Residual [0.00493097] Call [441] Scaled Residual [0.00493097] Call [442] Scaled Residual [0.00493097] Call [443] Scaled Residual [0.00493097] Call [444] Scaled Residual [0.00493097] Call [445] Scaled Residual [0.00493097] Call [446] Scaled Residual [0.00493097] Call [447] Scaled Residual [0.00493097] Call [448] Scaled Residual [0.00493097] Call [449] Scaled Residual [0.00493097] Call [450] Scaled Residual [0.00493097] Call [451] Scaled Residual [0.00493097] Call [452] Scaled Residual [0.00493097] Call [453] Scaled Residual [0.00493097] Call [454] Scaled Residual [0.00493097] Call [455] Scaled Residual [0.00493097] Call [456] Scaled Residual [0.00493097] Call [457] Scaled Residual [0.00493097] Call [458] Scaled Residual [0.00493097] Call [459] Scaled Residual [0.00493097] Call [460] Scaled Residual [0.00493097] Call [461] Scaled Residual [0.00493097] Call [462] Scaled Residual [0.00493097] Call [463] Scaled Residual [0.00493097] Call [464] Scaled Residual [0.00493097] Call [465] Scaled Residual [0.00493097] Call [466] Scaled Residual [0.00493097] Call [467] Scaled Residual [0.00493097] Call [468] Scaled Residual [0.00493097] Call [469] Scaled Residual [0.00493097] Call [470] Scaled Residual [0.00493097] Call [471] Scaled Residual [0.00493097] Call [472] Scaled Residual [0.00493097] Call [473] Scaled Residual [0.00493097] Call [474] Scaled Residual [0.00493097] Call [475] Scaled Residual [0.00493097] Call [476] Scaled Residual [0.00493097] Call [477] Scaled Residual [0.00493097] Call [478] Scaled Residual [0.00493097] Call [479] Scaled Residual [0.00493097] Call [480] Scaled Residual [0.00493097] Call [481] Scaled Residual [0.00493097] Call [482] Scaled Residual [0.00493097] Call [483] Scaled Residual [0.00493097] Call [484] Scaled Residual [0.00493097] Call [485] Scaled Residual [0.00493097] Call [486] Scaled Residual [0.00493097] Call [487] Scaled Residual [0.00493097] Call [488] Scaled Residual [0.00493097] Call [489] Scaled Residual [0.00493097] Call [490] Scaled Residual [0.00493097] Call [491] Scaled Residual [0.00493097] Call [492] Scaled Residual [0.00493097] Call [493] Scaled Residual [0.00493097] Call [494] Scaled Residual [0.00493097] Call [495] Scaled Residual [0.00493097] Call [496] Scaled Residual [0.00493097] Call [497] Scaled Residual [0.00493097] Call [498] Scaled Residual [0.00493097] Call [499] Scaled Residual [0.00493097] Call [500] Scaled Residual [0.00493097] Call [501] Scaled Residual [0.00493097] Call [502] Scaled Residual [0.00493097] Call [503] Scaled Residual [0.00493097] Call [504] Scaled Residual [0.00493097] Call [505] Scaled Residual [0.00493097] Call [506] Scaled Residual [0.00493097] Call [507] Scaled Residual [0.00493097] Call [508] Scaled Residual [0.00493097] Call [509] Scaled Residual [0.00493097] Call [510] Scaled Residual [0.00493097] Call [511] Scaled Residual [0.00493097] Call [512] Scaled Residual [0.00493097] Call [513] Scaled Residual [0.00493097] Call [514] Scaled Residual [0.00493097] Call [515] Scaled Residual [0.00493097] Call [516] Scaled Residual [0.00493097] Call [517] Scaled Residual [0.00493097] Call [518] Scaled Residual [0.00493097] Call [519] Scaled Residual [0.00493097] Call [520] Scaled Residual [0.00493097] Call [521] Scaled Residual [0.00493097] Call [522] Scaled Residual [0.00493097] Call [523] Scaled Residual [0.00493097] Call [524] Scaled Residual [0.00493097] Call [525] Scaled Residual [0.00493097] Call [526] Scaled Residual [0.00493097] Call [527] Scaled Residual [0.00493097] Call [528] Scaled Residual [0.00493097] Call [529] Scaled Residual [0.00493097] Call [530] Scaled Residual [0.00493097] Call [531] Scaled Residual [0.00493097] Call [532] Scaled Residual [0.00493097] Call [533] Scaled Residual [0.00493097] Call [534] Scaled Residual [0.00493097] Call [535] Scaled Residual [0.00493097] Call [536] Scaled Residual [0.00493097] Call [537] Scaled Residual [0.00493097] Call [538] Scaled Residual [0.00493097] Call [539] Scaled Residual [0.00493097] Call [540] Scaled Residual [0.00493097] Call [541] Scaled Residual [0.00493097] Call [542] Scaled Residual [0.00493097] Call [543] Scaled Residual [0.00493097] Call [544] Scaled Residual [0.00493097] Call [545] Scaled Residual [0.00493097] Call [546] Scaled Residual [0.00493097] Call [547] Scaled Residual [0.00493097] Call [548] Scaled Residual [0.00493097] Call [549] Scaled Residual [0.00493097] Call [550] Scaled Residual [0.00493097] Call [551] Scaled Residual [0.00493097] Call [552] Scaled Residual [0.00493097] Call [553] Scaled Residual [0.00493097] Call [554] Scaled Residual [0.00493097] Call [555] Scaled Residual [0.00493097] Call [556] Scaled Residual [0.00493097] Call [557] Scaled Residual [0.00493097] Call [558] Scaled Residual [0.00493097] Call [559] Scaled Residual [0.00493097] Call [560] Scaled Residual [0.00493097] Call [561] Scaled Residual [0.00493097] Call [562] Scaled Residual [0.00493097] Call [563] Scaled Residual [0.00493097] Call [564] Scaled Residual [0.00493097] Call [565] Scaled Residual [0.00493097] Call [566] Scaled Residual [0.00493097] Call [567] Scaled Residual [0.00493097] Call [568] Scaled Residual [0.00493097] Call [569] Scaled Residual [0.00493097] Call [570] Scaled Residual [0.00493097] Call [571] Scaled Residual [0.00493097] Call [572] Scaled Residual [0.00493097] Call [573] Scaled Residual [0.00493097] Call [574] Scaled Residual [0.00493097] Call [575] Scaled Residual [0.00493097] Call [576] Scaled Residual [0.00493097] Call [577] Scaled Residual [0.00493097] Call [578] Scaled Residual [0.00493097] Call [579] Scaled Residual [0.00493097] Call [580] Scaled Residual [0.00493097] Call [581] Scaled Residual [0.00493097] Call [582] Scaled Residual [0.00493097] Call [583] Scaled Residual [0.00493097] Call [584] Scaled Residual [0.00493097] Call [585] Scaled Residual [0.00493097] Call [586] Scaled Residual [0.00493097] Call [587] Scaled Residual [0.00493097] Call [588] Scaled Residual [0.00493097] Call [589] Scaled Residual [0.00493097] Call [590] Scaled Residual [0.00493097] Call [591] Scaled Residual [0.00493097] Call [592] Scaled Residual [0.00493097] Call [593] Scaled Residual [0.00493097] Call [594] Scaled Residual [0.00493097] Call [595] Scaled Residual [0.00493097] Call [596] Scaled Residual [0.00493097] Call [597] Scaled Residual [0.00493097] Call [598] Scaled Residual [0.00493097] Call [599] Scaled Residual [0.00493097] Call [600] Scaled Residual [0.00493097] Call [601] Scaled Residual [0.00493097] Call [602] Scaled Residual [0.00493097] Call [603] Scaled Residual [0.00493097] Call [604] Scaled Residual [0.00493097] Call [605] Scaled Residual [0.00493097] Call [606] Scaled Residual [0.00493097] Call [607] Scaled Residual [0.00493097] Call [608] Scaled Residual [0.00493097] Call [609] Scaled Residual [0.00493097] Call [610] Scaled Residual [0.00493097] Call [611] Scaled Residual [0.00493097] Call [612] Scaled Residual [0.00493097] Call [613] Scaled Residual [0.00493097] Call [614] Scaled Residual [0.00493097] Call [615] Scaled Residual [0.00493097] Call [616] Scaled Residual [0.00493097] Call [617] Scaled Residual [0.00493097] Call [618] Scaled Residual [0.00493097] Call [619] Scaled Residual [0.00493097] Call [620] Scaled Residual [0.00493097] Call [621] Scaled Residual [0.00493097] Call [622] Scaled Residual [0.00493097] Call [623] Scaled Residual [0.00493097] Call [624] Scaled Residual [0.00493097] Call [625] Scaled Residual [0.00493097] Call [626] Scaled Residual [0.00493097] Call [627] Scaled Residual [0.00493097] Call [628] Scaled Residual [0.00493097] Call [629] Scaled Residual [0.00493097] Call [630] Scaled Residual [0.00493097] Call [631] Scaled Residual [0.00493097] Call [632] Scaled Residual [0.00493097] Call [633] Scaled Residual [0.00493097] Call [634] Scaled Residual [0.00493097] Call [635] Scaled Residual [0.00493097] Call [636] Scaled Residual [0.00493097] Call [637] Scaled Residual [0.00493097] Call [638] Scaled Residual [0.00493097] Call [639] Scaled Residual [0.00493097] Call [640] Scaled Residual [0.00493097] Call [641] Scaled Residual [0.00493097] Call [642] Scaled Residual [0.00493097] Call [643] Scaled Residual [0.00493097] Call [644] Scaled Residual [0.00493097] Call [645] Scaled Residual [0.00493097] Call [646] Scaled Residual [0.00493097] Call [647] Scaled Residual [0.00493097] Call [648] Scaled Residual [0.00493097] Call [649] Scaled Residual [0.00493097] Call [650] Scaled Residual [0.00493097] Call [651] Scaled Residual [0.00493097] Call [652] Scaled Residual [0.00493097] Call [653] Scaled Residual [0.00493097] Call [654] Scaled Residual [0.00493097] Call [655] Scaled Residual [0.00493097] Call [656] Scaled Residual [0.00493097] Call [657] Scaled Residual [0.00493097] Call [658] Scaled Residual [0.00493097] Call [659] Scaled Residual [0.00493097] Call [660] Scaled Residual [0.00493097] Call [661] Scaled Residual [0.00493097] Call [662] Scaled Residual [0.00493097] Call [663] Scaled Residual [0.00493097] Call [664] Scaled Residual [0.00493097] Call [665] Scaled Residual [0.00493097] Call [666] Scaled Residual [0.00493097] Call [667] Scaled Residual [0.00493097] Call [668] Scaled Residual [0.00493097] Call [669] Scaled Residual [0.00493097] Call [670] Scaled Residual [0.00493097] Call [671] Scaled Residual [0.00493097] Call [672] Scaled Residual [0.00493097] Call [673] Scaled Residual [0.00493097] Call [674] Scaled Residual [0.00493097] Call [675] Scaled Residual [0.00493097] Call [676] Scaled Residual [0.00493097] Call [677] Scaled Residual [0.00493097] Call [678] Scaled Residual [0.00493097] Call [679] Scaled Residual [0.00493097] Call [680] Scaled Residual [0.00493097] Call [681] Scaled Residual [0.00493097] Call [682] Scaled Residual [0.00493097] Call [683] Scaled Residual [0.00493097] Call [684] Scaled Residual [0.00493097] Call [685] Scaled Residual [0.00493097] Call [686] Scaled Residual [0.00493097] Call [687] Scaled Residual [0.00493097] Call [688] Scaled Residual [0.00493097] Call [689] Scaled Residual [0.00493097] Call [690] Scaled Residual [0.00493097] Call [691] Scaled Residual [0.00493097] Call [692] Scaled Residual [0.00493097] Call [693] Scaled Residual [0.00493097] Call [694] Scaled Residual [0.00493097] Call [695] Scaled Residual [0.00493097] Call [696] Scaled Residual [0.00493097] Call [697] Scaled Residual [0.00493097] Call [698] Scaled Residual [0.00493097] Call [699] Scaled Residual [0.00493097] Call [700] Scaled Residual [0.00493097] Call [701] Scaled Residual [0.00493097] Call [702] Scaled Residual [0.00493097] Call [703] Scaled Residual [0.00493097] Call [704] Scaled Residual [0.00493097] Call [705] Scaled Residual [0.00493097] Call [706] Scaled Residual [0.00493097] Call [707] Scaled Residual [0.00493097] Call [708] Scaled Residual [0.00493097] Call [709] Scaled Residual [0.00493097] Call [710] Scaled Residual [0.00493097] Call [711] Scaled Residual [0.00493097] Call [712] Scaled Residual [0.00493097] Call [713] Scaled Residual [0.00493097] Call [714] Scaled Residual [0.00493097] Call [715] Scaled Residual [0.00493097] Call [716] Scaled Residual [0.00493097] Call [717] Scaled Residual [0.00493097] Call [718] Scaled Residual [0.00493097] Call [719] Scaled Residual [0.00493097] Call [720] Scaled Residual [0.00493097] Call [721] Scaled Residual [0.00493097] Call [722] Scaled Residual [0.00493097] Call [723] Scaled Residual [0.00493097] Call [724] Scaled Residual [0.00493097] Call [725] Scaled Residual [0.00493097] Call [726] Scaled Residual [0.00493097] Call [727] Scaled Residual [0.00493097] Call [728] Scaled Residual [0.00493097] Call [729] Scaled Residual [0.00493097] Call [730] Scaled Residual [0.00493097] Call [731] Scaled Residual [0.00493097] Call [732] Scaled Residual [0.00493097] Call [733] Scaled Residual [0.00493097] Call [734] Scaled Residual [0.00493097] Call [735] Scaled Residual [0.00493097] Call [736] Scaled Residual [0.00493097] Call [737] Scaled Residual [0.00493097] Call [738] Scaled Residual [0.00493097] Call [739] Scaled Residual [0.00493097] Call [740] Scaled Residual [0.00493097] Call [741] Scaled Residual [0.00493097] Call [742] Scaled Residual [0.00493097] Call [743] Scaled Residual [0.00493097] Call [744] Scaled Residual [0.00493097] Call [745] Scaled Residual [0.00493097] Call [746] Scaled Residual [0.00493097] Call [747] Scaled Residual [0.00493097] Call [748] Scaled Residual [0.00493097] Call [749] Scaled Residual [0.00493097] Call [750] Scaled Residual [0.00493097] Call [751] Scaled Residual [0.00493097] Call [752] Scaled Residual [0.00493097] Call [753] Scaled Residual [0.00493097] Call [754] Scaled Residual [0.00493097] Call [755] Scaled Residual [0.00493097] Call [756] Scaled Residual [0.00493097] Call [757] Scaled Residual [0.00493097] Call [758] Scaled Residual [0.00493097] Call [759] Scaled Residual [0.00493097] Call [760] Scaled Residual [0.00493097] Call [761] Scaled Residual [0.00493097] Call [762] Scaled Residual [0.00493097] Call [763] Scaled Residual [0.00493097] Call [764] Scaled Residual [0.00493097] Call [765] Scaled Residual [0.00493097] Call [766] Scaled Residual [0.00493097] Call [767] Scaled Residual [0.00493097] Call [768] Scaled Residual [0.00493097] Call [769] Scaled Residual [0.00493097] Call [770] Scaled Residual [0.00493097] Call [771] Scaled Residual [0.00493097] Call [772] Scaled Residual [0.00493097] Call [773] Scaled Residual [0.00493097] Call [774] Scaled Residual [0.00493097] Call [775] Scaled Residual [0.00493097] Call [776] Scaled Residual [0.00493097] Call [777] Scaled Residual [0.00493097] Call [778] Scaled Residual [0.00493097] Call [779] Scaled Residual [0.00493097] Call [780] Scaled Residual [0.00493097] Call [781] Scaled Residual [0.00493097] Call [782] Scaled Residual [0.00493097] Call [783] Scaled Residual [0.00493097] HPCG-Benchmark version=3.1 Release date=March 28, 2019 Machine Summary= Machine Summary::Distributed Processes=1440 Machine Summary::Threads per processes=14 Global Problem Dimensions= Global Problem Dimensions::Global nx=5120 Global Problem Dimensions::Global ny=6144 Global Problem Dimensions::Global nz=3072 Processor Dimensions= Processor Dimensions::npx=10 Processor Dimensions::npy=12 Processor Dimensions::npz=12 Local Domain Dimensions= Local Domain Dimensions::nx=512 Local Domain Dimensions::ny=512 ########## Problem Summary ##########= Setup Information= Setup Information::Setup Time=0.166606 Linear System Information= Linear System Information::Number of Equations=96636764160 Linear System Information::Number of Nonzero Terms=2608003719160 Multigrid Information= Multigrid Information::Number of coarse grid levels=3 Multigrid Information::Coarse Grids= Multigrid Information::Coarse Grids::Grid Level=1 Multigrid Information::Coarse Grids::Number of Equations=12079595520 Multigrid Information::Coarse Grids::Number of Nonzero Terms=325851893752 Multigrid Information::Coarse Grids::Number of Presmoother Steps=1 Multigrid Information::Coarse Grids::Number of Postsmoother Steps=1 Multigrid Information::Coarse Grids::Grid Level=2 Multigrid Information::Coarse Grids::Number of Equations=1509949440 Multigrid Information::Coarse Grids::Number of Nonzero Terms=40694360056 Multigrid Information::Coarse Grids::Number of Presmoother Steps=1 Multigrid Information::Coarse Grids::Number of Postsmoother Steps=1 Multigrid Information::Coarse Grids::Grid Level=3 Multigrid Information::Coarse Grids::Number of Equations=188743680 Multigrid Information::Coarse Grids::Number of Nonzero Terms=5077521400 Multigrid Information::Coarse Grids::Number of Presmoother Steps=1 Multigrid Information::Coarse Grids::Number of Postsmoother Steps=1 ########## Memory Use Summary ##########= Memory Use Information= Memory Use Information::Total memory used for data (Gbytes)=69081.4 Memory Use Information::Memory used for OptimizeProblem data (Gbytes)=0 Memory Use Information::Bytes per equation (Total memory / Number of Equations)=714.856 Memory Use Information::Memory used for linear system and CG (Gbytes)=60796.6 Memory Use Information::Coarse Grids= Memory Use Information::Coarse Grids::Grid Level=1 Memory Use Information::Coarse Grids::Memory used=7262.87 Memory Use Information::Coarse Grids::Grid Level=2 Memory Use Information::Coarse Grids::Memory used=908.242 Memory Use Information::Coarse Grids::Grid Level=3 Memory Use Information::Coarse Grids::Memory used=113.628 ########## V&V Testing Summary ##########= Spectral Convergence Tests= Spectral Convergence Tests::Result=PASSED Spectral Convergence Tests::Unpreconditioned= Spectral Convergence Tests::Unpreconditioned::Maximum iteration count=11 Spectral Convergence Tests::Unpreconditioned::Expected iteration count=12 Spectral Convergence Tests::Preconditioned= Spectral Convergence Tests::Preconditioned::Maximum iteration count=2 Spectral Convergence Tests::Preconditioned::Expected iteration count=2 Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon= Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon::Result=PASSED Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon::Departure for SpMV=0 Departure from Symmetry |x'Ay-y'Ax|/(2*||x||*||A||*||y||)/epsilon::Departure for MG=0 ########## Iterations Summary ##########= Iteration Count Information= Iteration Count Information::Result=PASSED Iteration Count Information::Reference CG iterations per set=50 Iteration Count Information::Optimized CG iterations per set=54 Iteration Count Information::Total number of reference iterations=39200 Iteration Count Information::Total number of optimized iterations=42336 ########## Reproducibility Summary ##########= Reproducibility Information= Reproducibility Information::Result=PASSED Reproducibility Information::Scaled residual mean=0.00493097 Reproducibility Information::Scaled residual variance=0 ########## Performance Summary (times in sec) ##########= Benchmark Time Summary= Benchmark Time Summary::Optimization phase=0.346134 Benchmark Time Summary::DDOT=66.1958 Benchmark Time Summary::WAXPBY=66.3085 Benchmark Time Summary::SpMV=330.171 Benchmark Time Summary::MG=1392.06 Benchmark Time Summary::Total=1854.76 Floating Point Operations Summary= Floating Point Operations Summary::Raw DDOT=2.46988e+16 Floating Point Operations Summary::Raw WAXPBY=2.46988e+16 Floating Point Operations Summary::Raw SpMV=2.24914e+17 Floating Point Operations Summary::Raw MG=1.26017e+18 Floating Point Operations Summary::Total=1.53448e+18 Floating Point Operations Summary::Total with convergence overhead=1.42081e+18 GB/s Summary= GB/s Summary::Raw Read B/W=5.09557e+06 GB/s Summary::Raw Write B/W=1.17774e+06 GB/s Summary::Raw Total B/W=6.27331e+06 GB/s Summary::Total with convergence and optimization phase overhead=5.6854e+06 GFLOP/s Summary= GFLOP/s Summary::Raw DDOT=373118 GFLOP/s Summary::Raw WAXPBY=372483 GFLOP/s Summary::Raw SpMV=681205 GFLOP/s Summary::Raw MG=905255 GFLOP/s Summary::Raw Total=827319 GFLOP/s Summary::Total with convergence overhead=766037 GFLOP/s Summary::Total with convergence and optimization phase overhead=749786 User Optimization Overheads= User Optimization Overheads::Optimization phase time (sec)=0.346134 User Optimization Overheads::Optimization phase time vs reference SpMV+MG time=0.0288874 DDOT Timing Variations= DDOT Timing Variations::Min DDOT MPI_Allreduce time=12.5765 DDOT Timing Variations::Max DDOT MPI_Allreduce time=26.0128 DDOT Timing Variations::Avg DDOT MPI_Allreduce time=16.368 Final Summary= Final Summary::HPCG result is VALID with a GFLOP/s rating of=749786 Final Summary::HPCG 2.4 rating for historical reasons is=754990 Final Summary::Please upload results from the YAML file contents to=http://hpcg-benchmark.org