Operating on Numpy arrays

Objectives

Introduce numpy’s pass-by-reference approach handling numpy arrays and methods for avoiding pitfalls when operating on numpy arrays.

Introduction

From: http://span.ece.utah.edu/common-mistakes-in-moving-from-matlab-to-python:

“Whenever you reference an array in Python, the computer will provide the memory address for the thing you are accessing, not the actual value. This is called pass-by-reference. This saves memory and makes your programs faster, but it is also harder to keep straight.”

From: https://docs.python.org/2/library/copy.html

“Assignment statements in Python do not copy objects, they create bindings [pointers] between a target and an object.” “… a copy is sometimes needed so one can change one copy without changing the other. The ‘copy’ module provides generic … copy operations.”

If you are not familiar with the pass-by-reference aspect of Python then I strongly suggest you read this short, informative essay on “Python Names and Values”: https://nedbatchelder.com/text/names.html

We’ve briefly touched on this important subject in earlier tutorials. Now we’ll go into a bit more detail.

Variable assignments

Unlike some other languages, creating a new variable with an assignment statement in Python such as x = some_numpy_array

does not make a copy of some_numpy_array. Instead, the assignment statement makes x and some_numpy_array both point to the same numpy array in memory. Because x and some_numpy_array are both refer (or pointer) to the same numpy array in memory, the numpy array can be changed by operations on either x or some_numpy_array. If you aren’t aware of this behavior then you may run into very difficult to identify bugs in your calculations!

A simple demonstration

Let’s demonstrate this issue with a very simple numpy array

[1]:
import numpy as np
import xarray as xr
import sys
import matplotlib.pyplot as plt
%matplotlib inline
import json
from copy import deepcopy
import warnings
warnings.filterwarnings('ignore')
[2]:
## Import the ecco_v4_py library into Python
## =========================================

## -- If ecco_v4_py is not installed in your local Python library,
##    tell Python where to find it.  For example, if your ecco_v4_py
##    files are in /Users/ifenty/ECCOv4-py/ecco_v4_py, then use:

sys.path.append('/home/ifenty/ECCOv4-py')
import ecco_v4_py as ecco
[3]:
## Set top-level file directory for the ECCO NetCDF files
## =================================================================
# base_dir = '/home/username/'
base_dir = '/home/ifenty/ECCOv4-release'

## define a high-level directory for ECCO fields
ECCO_dir = base_dir + '/Release3_alt'

Create a simple numpy array

[4]:
a=np.array([1, 2, 3, 4, 5])

# Assign 'b' to point to the same numpy array
b=a

# Test to see if b and a point to the same thing
b is a
[4]:
True

Now change the fourth element of b and print both a and b

[5]:
b[3] = 10
print (a)
print (b)
[ 1  2  3 10  5]
[ 1  2  3 10  5]

A fancier demonstration

Let’s now demonstrate with a numpy array that stores SSH output.

[6]:
## LOAD NETCDF SSH FILE

# directory of the file
data_dir= ECCO_dir + '/nctiles_monthly/SSH/'
# filename
fname = 'SSH_2010.nc'
# load the dataset file
ssh_dataset = xr.open_dataset(data_dir + fname).load()

## Load the model grid
grid_dir= ECCO_dir + '/nctiles_grid/'

ecco_grid = ecco.load_ecco_grid_nc(grid_dir, 'ECCOv4r3_grid.nc')

## Merge SSH and GRID
output_all = xr.merge((ssh_dataset, ecco_grid))

Recall the dimensions of our SSH DataArray:

[7]:
output_all.SSH.dims
[7]:
('time', 'tile', 'j', 'i')

Show the first four SSH values in j and i for the fifth month (May 1992) and second tile:

[8]:
output_all.SSH[4,1,0:4,0:4].values
[8]:
array([[-1.4365957, -1.436502 , -1.4301504, -1.434309 ],
       [-1.3933921, -1.3902488, -1.3870735, -1.401771 ],
       [-1.3574185, -1.3534354, -1.3570538, -1.3828031],
       [-1.3392653, -1.3360693, -1.3463365, -1.378602 ]], dtype=float32)

Assign the variable ssh_tmp to this subset of the numpy array that SSH points to:

[9]:
ssh_tmp = output_all.SSH[4,1,0:2,0:2].values
ssh_tmp
[9]:
array([[-1.4365957, -1.436502 ],
       [-1.3933921, -1.3902488]], dtype=float32)

Now change the values of all elements of ssh_tmp to 10

[10]:
ssh_tmp[:] = 10
ssh_tmp
[10]:
array([[10., 10.],
       [10., 10.]], dtype=float32)

And see that yes, in fact, this change is reflected in our SSH DataArray:

[11]:
output_all.SSH[4,1,0:4,0:4].values
[11]:
array([[10.       , 10.       , -1.4301504, -1.434309 ],
       [10.       , 10.       , -1.3870735, -1.401771 ],
       [-1.3574185, -1.3534354, -1.3570538, -1.3828031],
       [-1.3392653, -1.3360693, -1.3463365, -1.378602 ]], dtype=float32)

Dealing with pass-by-reference: right hand side operations

One way to have a new variable assignment not point to the original variable is to perform an operation on the right hand side of the assignment statement.

“Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.” https://docs.python.org/2/reference/expressions.html#evaluation-order

Performing an operation on the right hand side creates new values in memory. The new variable assignment will then point to these new values, leaving the original untouched.

Simple demonstration 1

Operate on a by adding 1 before the assigment statement

[12]:
# Create a simple numpy array
a=np.array([1, 2, 3, 4, 5])

b = a + 1

print (a)
print (b)
[1 2 3 4 5]
[2 3 4 5 6]

Now change the fourth element of b and print both a and b

[13]:
b[3] = 10
print (a)
print (b)
[1 2 3 4 5]
[ 2  3  4 10  6]

a and b do indeed point to different values in memory.

Simple demonstration 2

Operate on a by adding 0 before the assigment statement. This is a kind of dummy operation.

[14]:
# Create a simple numpy array
a=np.array([1, 2, 3, 4, 5])

# Add 0 to `a`:
b = a + 0

print (a)
print (b)
[1 2 3 4 5]
[1 2 3 4 5]
[15]:
# Test to see if b and a point to the same thing
b is a
[15]:
False

Now change the fourth element of b and print both a and b

[16]:
b[3] = 10
print (a)
print (b)
[1 2 3 4 5]
[ 1  2  3 10  5]

Once again we see that a and b do indeed point to different values in memory.

A fancier demonstration

Let’s now demonstrate with a numpy array that stores SSH output.

[17]:
output_all.SSH[4,1,5:9,5:9].values
[17]:
array([[-1.4406753, -1.4481026, -1.4435667, -1.4330329],
       [-1.4137946, -1.4145803, -1.4049884, -1.3896267],
       [-1.3741281, -1.3701942, -1.3568683, -1.3380871],
       [-1.3235712, -1.316149 , -1.2999685, -1.2789109]], dtype=float32)
[18]:
ssh_tmp = output_all.SSH[4,1,5:9,5:9].values * output_all.rA[1,5:9,5:9].values
ssh_tmp[:] = 10
ssh_tmp
[18]:
array([[10., 10., 10., 10.],
       [10., 10., 10., 10.],
       [10., 10., 10., 10.],
       [10., 10., 10., 10.]], dtype=float32)
[19]:
output_all.SSH[4,1,5:9,5:9].values
[19]:
array([[-1.4406753, -1.4481026, -1.4435667, -1.4330329],
       [-1.4137946, -1.4145803, -1.4049884, -1.3896267],
       [-1.3741281, -1.3701942, -1.3568683, -1.3380871],
       [-1.3235712, -1.316149 , -1.2999685, -1.2789109]], dtype=float32)

Operating on the right hand side of the assignment does indeed new arrays in memory leaving the original SSH numpy array untouched.

Dealing with pass-by-reference: copy and deepcopy

A second way to have a new variable assignment not point to the original variable is to use the copy or deepcopy command.

Simple demonstration

Use the numpy command.

[20]:
# Create a simple numpy array
a=np.array([1, 2, 3, 4, 5])
b=np.copy(a)

print (a)
print (b)
[1 2 3 4 5]
[1 2 3 4 5]

Now change the fourth element of b and print both a and b

[21]:
b[3] = 10
print (a)
print (b)
[1 2 3 4 5]
[ 1  2  3 10  5]
[22]:
output_all.SSH
[22]:
<xarray.DataArray 'SSH' (time: 12, tile: 13, j: 90, i: 90)>
array([[[[ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.4836141 , -1.4890583 , -1.4887266 , ..., -1.3676528 ,
          -1.3607242 , -1.3533412 ],
         [-1.4628036 , -1.468267  , -1.4667836 , ..., -1.3557527 ,
          -1.3482903 , -1.3397818 ],
         [-1.436315  , -1.4399437 , -1.4370856 , ..., -1.3461887 ,
          -1.3382714 , -1.3282658 ]],

        [[-1.4029788 , -1.4037812 , -1.4008827 , ..., -1.3376493 ,
          -1.3292958 , -1.3175498 ],
         [-1.3666646 , -1.365299  , -1.3648341 , ..., -1.3276558 ,
          -1.3189425 , -1.3056219 ],
         [-1.33584   , -1.334351  , -1.3388703 , ..., -1.3141139 ,
          -1.3052138 , -1.2908865 ],
         ...,
         [ 0.14946148,  0.1455971 ,  0.14140229, ...,  0.        ,
           0.4259282 ,  0.40249246],
         [ 0.13342337,  0.1310458 ,  0.12794277, ...,  0.        ,
           0.42839238,  0.41154262],
         [ 0.11906417,  0.11801734,  0.11567361, ...,  0.        ,
           0.        ,  0.41444954]],

        [[ 0.10849833,  0.10763063,  0.10525029, ...,  0.        ,
           0.        ,  0.41118386],
         [ 0.10236199,  0.1005336 ,  0.09722137, ...,  0.31839028,
           0.        ,  0.40181533],
         [ 0.10205435,  0.09833906,  0.09325342, ...,  0.353757  ,
           0.35896647,  0.40689626],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ]],

        ...,

        [[ 0.        ,  0.        ,  0.        , ...,  0.35200754,
           0.326652  ,  0.30624378],
         [ 0.        ,  0.        ,  0.        , ...,  0.35387242,
           0.32897007,  0.3091959 ],
         [ 0.        ,  0.        ,  0.        , ...,  0.3596493 ,
           0.33368325,  0.31345782],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.11090944,
           0.10653148,  0.1093755 ],
         [ 0.        ,  0.        ,  0.        , ...,  0.10750817,
           0.1044946 ,  0.10859182],
         [ 0.        ,  0.        ,  0.        , ...,  0.10480525,
           0.10333875,  0.10848776]],

        [[ 0.30412942,  0.33163443,  0.388738  , ..., -0.35260823,
          -0.42698237, -0.50772387],
         [ 0.30684927,  0.3322026 ,  0.386021  , ..., -0.31668594,
          -0.38891688, -0.4718595 ],
         [ 0.3105247 ,  0.33349368,  0.38373435, ..., -0.3030887 ,
          -0.37445134, -0.45894527],
         ...,
         [ 0.11961964,  0.13626505,  0.15554132, ..., -1.2845815 ,
          -1.3033888 , -1.3298517 ],
         [ 0.11943392,  0.13602602,  0.15484709, ..., -1.3105577 ,
          -1.3325728 , -1.3604729 ],
         [ 0.11928844,  0.1349795 ,  0.15260401, ..., -1.3288026 ,
          -1.3561577 , -1.3883135 ]],

        [[-0.5964835 , -0.69236034, -0.79338306, ...,  0.        ,
           0.        ,  0.        ],
         [-0.5660535 , -0.6684363 , -0.7742219 , ...,  0.        ,
           0.        ,  0.        ],
         [-0.5563829 , -0.66219753, -0.7694144 , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.361862  , -1.3975031 , -1.4352301 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.3903762 , -1.4201684 , -1.4501717 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.4191432 , -1.4458475 , -1.4696381 , ...,  0.        ,
           0.        ,  0.        ]]],


       [[[ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.489962  , -1.49527   , -1.4950125 , ..., -1.324543  ,
          -1.31546   , -1.3052927 ],
         [-1.4686763 , -1.474007  , -1.4730073 , ..., -1.3154641 ,
          -1.3059864 , -1.2944871 ],
         [-1.4415836 , -1.445527  , -1.4437983 , ..., -1.3081948 ,
          -1.2985847 , -1.2857456 ]],

        [[-1.4074324 , -1.4091369 , -1.4077874 , ..., -1.3005854 ,
          -1.2910386 , -1.2770216 ],
         [-1.3706107 , -1.3701998 , -1.3714415 , ..., -1.2901691 ,
          -1.2808572 , -1.2660264 ],
         [-1.3400941 , -1.3389751 , -1.3451121 , ..., -1.2752724 ,
          -1.2662243 , -1.2510723 ],
         ...,
         [ 0.14600152,  0.14037795,  0.13405259, ...,  0.        ,
           0.45676908,  0.46960458],
         [ 0.13392949,  0.12819704,  0.12193992, ...,  0.        ,
           0.43185055,  0.44771263],
         [ 0.12269467,  0.11731683,  0.11124787, ...,  0.        ,
           0.        ,  0.42728487]],

        [[ 0.11415915,  0.10866896,  0.10236249, ...,  0.        ,
           0.        ,  0.41325122],
         [ 0.1091438 ,  0.10260367,  0.09539554, ...,  0.34004778,
           0.        ,  0.40250835],
         [ 0.10801908,  0.10005993,  0.09191936, ...,  0.370763  ,
           0.37222847,  0.40804937],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ]],

        ...,

        [[ 0.        ,  0.        ,  0.        , ...,  0.39924735,
           0.38143918,  0.36281937],
         [ 0.        ,  0.        ,  0.        , ...,  0.4045941 ,
           0.38406068,  0.36293516],
         [ 0.        ,  0.        ,  0.        , ...,  0.40966666,
           0.38610974,  0.36248463],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.12320995,
           0.12149823,  0.12537551],
         [ 0.        ,  0.        ,  0.        , ...,  0.11993366,
           0.11874654,  0.12275659],
         [ 0.        ,  0.        ,  0.        , ...,  0.11488416,
           0.11462326,  0.11893921]],

        [[ 0.35339177,  0.36148992,  0.39120036, ..., -0.3515566 ,
          -0.4434473 , -0.5372167 ],
         [ 0.35098886,  0.35647574,  0.384084  , ..., -0.31312683,
          -0.40090024, -0.49447078],
         [ 0.34828657,  0.35177138,  0.37818742, ..., -0.29459885,
          -0.3796233 , -0.4731894 ],
         ...,
         [ 0.13400438,  0.14537305,  0.15664023, ..., -1.2893282 ,
          -1.3100575 , -1.3397995 ],
         [ 0.13128757,  0.14274582,  0.15433277, ..., -1.3145573 ,
          -1.3370266 , -1.3671248 ],
         [ 0.1274146 ,  0.1388315 ,  0.15071633, ..., -1.3328918 ,
          -1.359795  , -1.3930436 ]],

        [[-0.63250536, -0.72816867, -0.82351893, ...,  0.        ,
           0.        ,  0.        ],
         [-0.59260696, -0.6927064 , -0.79215497, ...,  0.        ,
           0.        ,  0.        ],
         [-0.5730935 , -0.6754043 , -0.7760804 , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.3741646 , -1.4095769 , -1.4447788 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.3989064 , -1.4287863 , -1.457388  , ...,  0.        ,
           0.        ,  0.        ],
         [-1.4253014 , -1.4526149 , -1.4762621 , ...,  0.        ,
           0.        ,  0.        ]]],


       [[[ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.5110403 , -1.5177205 , -1.5174176 , ..., -1.3357865 ,
          -1.3278884 , -1.318876  ],
         [-1.4881549 , -1.4948115 , -1.4935229 , ..., -1.3228539 ,
          -1.314229  , -1.3036441 ],
         [-1.4580069 , -1.4625804 , -1.4603127 , ..., -1.3126371 ,
          -1.303386  , -1.290982  ]],

        [[-1.4192245 , -1.4211298 , -1.4195518 , ..., -1.3029585 ,
          -1.2933172 , -1.2792544 ],
         [-1.3780817 , -1.3780969 , -1.3797989 , ..., -1.2915645 ,
          -1.2817683 , -1.2664136 ],
         [-1.3462648 , -1.3455915 , -1.3522567 , ..., -1.2767069 ,
          -1.2668498 , -1.2507555 ],
         ...,
         [ 0.1566297 ,  0.14574713,  0.1340149 , ...,  0.        ,
           0.5034744 ,  0.52975076],
         [ 0.14160103,  0.13072166,  0.11933897, ...,  0.        ,
           0.46551415,  0.49194607],
         [ 0.12809302,  0.1176656 ,  0.10695522, ...,  0.        ,
           0.        ,  0.45891845]],

        [[ 0.11827811,  0.10802748,  0.09787346, ...,  0.        ,
           0.        ,  0.4385822 ],
         [ 0.11309812,  0.10250971,  0.09244011, ...,  0.3657641 ,
           0.        ,  0.42507946],
         [ 0.11333717,  0.10238108,  0.09199497, ...,  0.3817734 ,
           0.38925597,  0.42275748],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ]],

        ...,

        [[ 0.        ,  0.        ,  0.        , ...,  0.42629334,
           0.40751386,  0.38404158],
         [ 0.        ,  0.        ,  0.        , ...,  0.42555302,
           0.40451616,  0.37930268],
         [ 0.        ,  0.        ,  0.        , ...,  0.42385182,
           0.40150982,  0.37547565],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.14473848,
           0.14240266,  0.14556926],
         [ 0.        ,  0.        ,  0.        , ...,  0.13558038,
           0.13387772,  0.13747904],
         [ 0.        ,  0.        ,  0.        , ...,  0.12472015,
           0.12382509,  0.12815464]],

        [[ 0.36552784,  0.35883728,  0.36747   , ..., -0.37859654,
          -0.4690925 , -0.5647183 ],
         [ 0.35887176,  0.35017943,  0.35735488, ..., -0.34685484,
          -0.4315016 , -0.5234984 ],
         [ 0.35353845,  0.34268492,  0.34815198, ..., -0.33039528,
          -0.41046697, -0.4991011 ],
         ...,
         [ 0.15426582,  0.16672038,  0.17900392, ..., -1.2917107 ,
          -1.3121684 , -1.3426142 ],
         [ 0.1466583 ,  0.15988627,  0.17363326, ..., -1.3188969 ,
          -1.3416951 , -1.3739729 ],
         [ 0.13771701,  0.1512237 ,  0.16593452, ..., -1.3384157 ,
          -1.3662696 , -1.4031518 ]],

        [[-0.66363436, -0.76205665, -0.85687906, ...,  0.        ,
           0.        ,  0.        ],
         [-0.6209822 , -0.71981865, -0.8161234 , ...,  0.        ,
           0.        ,  0.        ],
         [-0.59452295, -0.692481  , -0.7885612 , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.3775544 , -1.4144093 , -1.4529849 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.4076394 , -1.439422  , -1.4712203 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.4387896 , -1.4683706 , -1.4941783 , ...,  0.        ,
           0.        ,  0.        ]]],


       ...,


       [[[ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.5169188 , -1.5242634 , -1.5220859 , ..., -1.36742   ,
          -1.3529748 , -1.3388332 ],
         [-1.4893768 , -1.4971659 , -1.4928439 , ..., -1.3538916 ,
          -1.3381528 , -1.3220502 ],
         [-1.4557267 , -1.4612318 , -1.4543641 , ..., -1.3420252 ,
          -1.3259686 , -1.308566  ]],

        [[-1.4152532 , -1.4169811 , -1.409366  , ..., -1.3315804 ,
          -1.3167756 , -1.2991067 ],
         [-1.3728914 , -1.3715063 , -1.3669327 , ..., -1.3211186 ,
          -1.3085409 , -1.2908047 ],
         [-1.3378468 , -1.3357688 , -1.3373411 , ..., -1.3085197 ,
          -1.298021  , -1.280042  ],
         ...,
         [ 0.14488927,  0.14486471,  0.1455704 , ...,  0.        ,
           0.41786316,  0.48173904],
         [ 0.11153719,  0.11205091,  0.11443915, ...,  0.        ,
           0.37641078,  0.44489366],
         [ 0.08118004,  0.08179028,  0.08559205, ...,  0.        ,
           0.        ,  0.40048105]],

        [[ 0.05805106,  0.05808871,  0.0618963 , ...,  0.        ,
           0.        ,  0.36893418],
         [ 0.04563557,  0.0444993 ,  0.04670291, ...,  0.23532906,
           0.        ,  0.33999464],
         [ 0.04870504,  0.04574053,  0.04486019, ...,  0.24327128,
           0.26102594,  0.32868192],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ]],

        ...,

        [[ 0.        ,  0.        ,  0.        , ...,  0.31560326,
           0.28781885,  0.30457547],
         [ 0.        ,  0.        ,  0.        , ...,  0.30183744,
           0.27608654,  0.29699475],
         [ 0.        ,  0.        ,  0.        , ...,  0.2909881 ,
           0.26693258,  0.29118684],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.07493891,
           0.07432082,  0.08609825],
         [ 0.        ,  0.        ,  0.        , ...,  0.06333499,
           0.061132  ,  0.07336642],
         [ 0.        ,  0.        ,  0.        , ...,  0.05451504,
           0.05124369,  0.06337053]],

        [[ 0.36829612,  0.45668772,  0.5346397 , ..., -0.5123356 ,
          -0.5917844 , -0.6731994 ],
         [ 0.36443853,  0.4552048 ,  0.5336968 , ..., -0.4702206 ,
          -0.54875636, -0.6336393 ],
         [ 0.36134198,  0.45349684,  0.53227144, ..., -0.4475117 ,
          -0.5257021 , -0.6136234 ],
         ...,
         [ 0.10326117,  0.12333801,  0.14608134, ..., -1.28159   ,
          -1.2933701 , -1.3159804 ],
         [ 0.09370542,  0.11881482,  0.1466319 , ..., -1.308702  ,
          -1.3279408 , -1.3554627 ],
         [ 0.08548156,  0.11409506,  0.14582734, ..., -1.3291799 ,
          -1.3578951 , -1.3934467 ]],

        [[-0.75963545, -0.8508026 , -0.9435896 , ...,  0.        ,
           0.        ,  0.        ],
         [-0.72582525, -0.8225674 , -0.9187593 , ...,  0.        ,
           0.        ,  0.        ],
         [-0.7104533 , -0.8112295 , -0.9089868 , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.3510643 , -1.3976614 , -1.4507179 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.3892317 , -1.4282138 , -1.4713119 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.4299054 , -1.4643455 , -1.4974834 , ...,  0.        ,
           0.        ,  0.        ]]],


       [[[ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.4967054 , -1.502654  , -1.5001634 , ..., -1.3340458 ,
          -1.3227609 , -1.31188   ],
         [-1.4694204 , -1.4754978 , -1.4709007 , ..., -1.3241014 ,
          -1.311319  , -1.2982643 ],
         [-1.4372219 , -1.440807  , -1.4335835 , ..., -1.3163857 ,
          -1.3029233 , -1.2881821 ]],

        [[-1.3995142 , -1.399057  , -1.3908552 , ..., -1.3096857 ,
          -1.2967411 , -1.2810549 ],
         [-1.359279  , -1.3553827 , -1.3499216 , ..., -1.3022889 ,
          -1.290798  , -1.274405  ],
         [-1.323668  , -1.3191642 , -1.3201401 , ..., -1.2925091 ,
          -1.2827281 , -1.2657163 ],
         ...,
         [ 0.11687395,  0.11706704,  0.11802986, ...,  0.        ,
           0.4334885 ,  0.44428313],
         [ 0.09929278,  0.10102028,  0.10379542, ...,  0.        ,
           0.41442624,  0.43558455],
         [ 0.08507469,  0.08816329,  0.0925502 , ...,  0.        ,
           0.        ,  0.4132457 ]],

        [[ 0.07387409,  0.077453  ,  0.08254188, ...,  0.        ,
           0.        ,  0.39164448],
         [ 0.06708062,  0.06982116,  0.07399624, ...,  0.24350804,
           0.        ,  0.3668178 ],
         [ 0.0685271 ,  0.06928899,  0.07084791, ...,  0.28005257,
           0.29600853,  0.36135092],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ]],

        ...,

        [[ 0.        ,  0.        ,  0.        , ...,  0.26727462,
           0.22990613,  0.23907676],
         [ 0.        ,  0.        ,  0.        , ...,  0.25806925,
           0.22213893,  0.23322569],
         [ 0.        ,  0.        ,  0.        , ...,  0.25158796,
           0.21656525,  0.22877966],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.08118027,
           0.07804877,  0.08201623],
         [ 0.        ,  0.        ,  0.        , ...,  0.07406381,
           0.07104729,  0.07628159],
         [ 0.        ,  0.        ,  0.        , ...,  0.06990036,
           0.06730542,  0.07331178]],

        [[ 0.29907313,  0.3912239 ,  0.48033908, ..., -0.47317785,
          -0.56365776, -0.65234643],
         [ 0.2947312 ,  0.3865718 ,  0.47382972, ..., -0.42516652,
          -0.5157564 , -0.61019945],
         [ 0.29063997,  0.38139656,  0.46688396, ..., -0.39321777,
          -0.4851443 , -0.58585995],
         ...,
         [ 0.08940418,  0.10047803,  0.11612792, ..., -1.2786794 ,
          -1.2925618 , -1.3149059 ],
         [ 0.08617538,  0.09981632,  0.11717496, ..., -1.301788  ,
          -1.3228261 , -1.3493527 ],
         [ 0.08436081,  0.09902733,  0.11718916, ..., -1.3183063 ,
          -1.3480992 , -1.381798  ]],

        [[-0.74166995, -0.83347535, -0.92762154, ...,  0.        ,
           0.        ,  0.        ],
         [-0.707929  , -0.8075284 , -0.906459  , ...,  0.        ,
           0.        ,  0.        ],
         [-0.69183034, -0.7981904 , -0.8999608 , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.348128  , -1.3918104 , -1.4414979 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.3810068 , -1.4177865 , -1.4585305 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.4156406 , -1.4483571 , -1.4804294 , ...,  0.        ,
           0.        ,  0.        ]]],


       [[[ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.463372  , -1.466484  , -1.4635477 , ..., -1.318022  ,
          -1.3103173 , -1.3024466 ],
         [-1.4383494 , -1.4415286 , -1.4376326 , ..., -1.3078314 ,
          -1.2983781 , -1.2879825 ],
         [-1.4094868 , -1.4109806 , -1.4058659 , ..., -1.3008653 ,
          -1.2901821 , -1.277352  ]],

        [[-1.3759882 , -1.3743817 , -1.3691081 , ..., -1.2955976 ,
          -1.2844023 , -1.2695681 ],
         [-1.3394302 , -1.3350021 , -1.3323509 , ..., -1.2904404 ,
          -1.2794557 , -1.2629302 ],
         [-1.3061111 , -1.301164  , -1.3042493 , ..., -1.2837274 ,
          -1.2732459 , -1.2554117 ],
         ...,
         [ 0.12598938,  0.1267241 ,  0.12829028, ...,  0.        ,
           0.3990867 ,  0.37388888],
         [ 0.1142365 ,  0.1159412 ,  0.11839783, ...,  0.        ,
           0.39991677,  0.3842097 ],
         [ 0.1103783 ,  0.11341098,  0.11685228, ...,  0.        ,
           0.        ,  0.3827097 ]],

        [[ 0.11015419,  0.11427843,  0.11837347, ...,  0.        ,
           0.        ,  0.37261513],
         [ 0.11017552,  0.11442809,  0.11824088, ...,  0.27746603,
           0.        ,  0.3570833 ],
         [ 0.11064535,  0.11360037,  0.11599879, ...,  0.32267517,
           0.3240342 ,  0.36032936],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ],
         [ 0.        ,  0.        ,  0.        , ...,  0.        ,
           0.        ,  0.        ]],

        ...,

        [[ 0.        ,  0.        ,  0.        , ...,  0.25312427,
           0.20679894,  0.19787078],
         [ 0.        ,  0.        ,  0.        , ...,  0.24408737,
           0.19740611,  0.18982588],
         [ 0.        ,  0.        ,  0.        , ...,  0.23627043,
           0.19088954,  0.18501939],
         ...,
         [ 0.        ,  0.        ,  0.        , ...,  0.11057193,
           0.10666864,  0.10647956],
         [ 0.        ,  0.        ,  0.        , ...,  0.10842457,
           0.10574125,  0.10616441],
         [ 0.        ,  0.        ,  0.        , ...,  0.10844468,
           0.10687705,  0.10721517]],

        [[ 0.23266444,  0.30498484,  0.39119166, ..., -0.45387354,
          -0.54551655, -0.63072217],
         [ 0.22718684,  0.30146563,  0.3875697 , ..., -0.39773405,
          -0.48800388, -0.5802953 ],
         [ 0.22384512,  0.2981879 ,  0.38286942, ..., -0.35758346,
          -0.4479277 , -0.547731  ],
         ...,
         [ 0.10817628,  0.11316824,  0.12408888, ..., -1.2703364 ,
          -1.2884878 , -1.312592  ],
         [ 0.10809258,  0.1135454 ,  0.1254746 , ..., -1.290234  ,
          -1.3133153 , -1.3399168 ],
         [ 0.10853176,  0.11351557,  0.12581852, ..., -1.3032938 ,
          -1.3325437 , -1.3641984 ]],

        [[-0.7145314 , -0.80162513, -0.89393973, ...,  0.        ,
           0.        ,  0.        ],
         [-0.67591274, -0.77460396, -0.87444943, ...,  0.        ,
           0.        ,  0.        ],
         [-0.6548358 , -0.76413405, -0.869508  , ...,  0.        ,
           0.        ,  0.        ],
         ...,
         [-1.343692  , -1.3808477 , -1.4207497 , ...,  0.        ,
           0.        ,  0.        ],
         [-1.3690091 , -1.4003408 , -1.43325   , ...,  0.        ,
           0.        ,  0.        ],
         [-1.394759  , -1.4236512 , -1.4513302 , ...,  0.        ,
           0.        ,  0.        ]]]], dtype=float32)
Coordinates:
  * j        (j) int32 0 1 2 3 4 5 6 7 8 9 10 ... 80 81 82 83 84 85 86 87 88 89
  * i        (i) int32 0 1 2 3 4 5 6 7 8 9 10 ... 80 81 82 83 84 85 86 87 88 89
    XC       (tile, j, i) float32 -111.60647 -111.303 ... -105.58465 -111.86579
    YC       (tile, j, i) float32 -88.24259 -88.382515 ... -88.07871 -88.10267
    rA       (tile, j, i) float32 362256450.0 363300960.0 ... 361119100.0
  * tile     (tile) int32 0 1 2 3 4 5 6 7 8 9 10 11 12
    iter     (time) int32 158532 159204 159948 160668 ... 165084 165804 166548
  * time     (time) datetime64[ns] 2010-01-16T12:00:00 ... 2010-12-16T12:00:00
    CS       (tile, j, i) float32 0.06157813 0.06675376 ... -0.9983638
    SN       (tile, j, i) float32 -0.99810225 -0.9977695 ... -0.057182025
    Depth    (tile, j, i) float32 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0 0.0 0.0 0.0
Attributes:
    units:          m
    long_name:      Surface Height Anomaly adjusted with global steric height...
    standard_name:  sea_surface_height

Fancier demonstration

Dataset and DataArray objects are too complicated for numpy’s copy command. For complex objects such as these use the deepcopy command.

[23]:
ssh_tmp = deepcopy(output_all.SSH)
ssh_tmp[:] = 10
ssh_tmp[4,1,5:9,5:9].values
[23]:
array([[10., 10., 10., 10.],
       [10., 10., 10., 10.],
       [10., 10., 10., 10.],
       [10., 10., 10., 10.]], dtype=float32)
[24]:
output_all.SSH[4,1,5:9,5:9].values
[24]:
array([[-1.4406753, -1.4481026, -1.4435667, -1.4330329],
       [-1.4137946, -1.4145803, -1.4049884, -1.3896267],
       [-1.3741281, -1.3701942, -1.3568683, -1.3380871],
       [-1.3235712, -1.316149 , -1.2999685, -1.2789109]], dtype=float32)

Using deepcopy gives us an entirely new array in memory. Operations on ssh_tmp do not affect the original fields that we found in the output_all_SSH DataArray.

alternative to deepcopy

xarray give us another way to deepcopy DataArrays and Datasets:

ssh_tmp = output_all.copy(deep=True)

Conclusion

You now know about the possible pitfalls for dealing with Python’s pass-by-reference way of handling assignment statements and different methods for making copies of numpy arrays and Datasets and DataArrays.