vtkIdList in ESRI ASCII Grid Reader
The vtk docs describe a way to use arrays in a vtkIDList. See if the python bindings allow this as well and try out if this brings any performance gain. https://vtk.org/doc/nightly/html/classvtkIdList.html#a791c137b70f53179bd9df65f93d954de