NumPy

NumPy#

What you need to know

NumPy is the core python library for numerical and scientific computing.
A numpy array is a grid of values, all of the same type, and is indexed by nonnegative integers.
The array can have any number of dimensions 1D, 2D, 3D, …
The shape of an array is a tuple of integers giving the size of the array along each dimension. For example a 1D vector of size 4 is (4,). a matrix of size 2 is (2,2), a matrix with size 2x5 is (2,5)
Numpy arrays can be generates either by feeding lists to numpy or on the fly using numpy special methods

There are many great resources out there to learn numpy. See these three

Array Creation#

Generating numpy arrays from lists#

a=np.array([1,2,3])
a

type(a)

data.shape

Just like lists we can change elements via assignment

Lists of lists create 2D arrays!

b = np.array([[1,2,3],[4,5,6]])   

b

b.shape

# Question: what does this do?
np.array( [ x**2 for x in range(100) if x%3==0 ])

Array vs list: which is faster?#

We can use %timit to compare speeds of elementwise operations done with lits vs numpy

%timeit [x**2 for x in range(10000)]

x = np.arange(10000)

%timeit x**2

Notice that numpy carried out squaring on every single element at once instead of requiring manual iteration.
With 10,000 integers, the Python list and for loop takes an average of single milliseconds, while the NumPy array completes the same operation in tens of microseconds. This is a speed increase of over 100x by using the NumPy array (1 millisecond = 1000 microseconds).
For larger lists of numbers, the speed increase using NumPy is considerable.

Vectorized operations with numpy#

Basic mathematical functions operate elementwise on arrays!
Example: np.sqrt(x) or x**0.5 will take square root of every single element on numpy array x

x = np.array([1,2,3,4])
y = np.array([5,6,7,8])

x+y

x*y

x/y

np.sqrt(y)

y**0.5

x+3

The addition example shows that one can also do operations on arrays with unequal shapes!
In mathematics you can’t add vector to a scalar but in numpy you can!
These are powerful operations are called broadcasting. See the end for these rules and examples

Generating arrays using special methods#

Creating arrays of ones or zeros can also be useful as placeholder arrays, in cases where we do not want to use the initial values for computations but want to fill it with other values right away.
For instance np.zeros, np.ones, np.empty create such placeholder arrays.
np.random contains many functions for generating random numbers. We will utilize those to build simulations
There are large set of methods for generating arrays for common numeric tasts. Below are listed a few we will use most etensively

Function	Description
`np.array([list, of, numbers])`	Array from a list
`np.arange(start, stop, step)`	Array with know step
`np.linspace(start, stop, num)`	Creates an array from [start, stop] with num number of steps
`np.zeros((rows, cols))`	Array of zeros
`np.ones((rows, cols))`	Array of ones
`np.meshgrid(array1, array2)`	Two 2D arrays from two 1D arrays
`np.rand()`	Generates random floats in the range [0,1) in an even distribution
`np.randn()`	Generates random floats in a normal distribution centered around zero

np.zeros(3)  # Create an array of all zeros

np.ones(11)

np.emtpy(3)

np.random.randn(5)

Genearting N-dimensional arrays#

Generating 1D arrays is done by specifying length N np.zeros(N)
Generating 2D arrays is done by specifying Nrows and M columns np.zeros((N, M))
Gemerating 3D arrays is done by specigyin

np.ones((1,5))   

np.zeros(2,2)

np.random.random((4,4)) # Create an array filled with random values

Indexing, slicing and shaping arrays#

Slicing: Similar to Python lists, numpy arrays can be sliced. Since arrays may be multidimensional, you must specify a slice for each dimension of the array:

data=np.array([1,2,3])
data[0:3]

Quick example: Create and Slice the data to get the elements shown

a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
a.shape

Predict the sliced elements

a.shape

a[1,:4]  #

a[1,3]

a[:,-1] 

a[-1,:] 

Boolean masks#

We can also use Boolean masks for indexing – that is, arrays of True and False values. Consider the following example, where we return all values in the array that are greater than 3:

ary = np.array([1, 2, 3, 4])
mask = ary > 2
mask

array([False, False,  True,  True])

ary[mask]

Or you can use the boolean mask directly on array

ary[ary<4]

array([1, 2, 3])

ary = np.array([[1, 2, 3],
                [4, 5, 6]])

ary[ary > 3]

A related, useful function to assign values to specific elements in an array is the np.where function. In the example below, we assign a 1 to all values in the array that are greater than 2 – and 0, otherwise:

np.where(ary > 2, 1, 0)

array([0, 0, 1, 1])

There are also so-called bit-wise operators that we can use to specify more complex selection criteria:

ary = np.array([1, 2, 3, 4])
mask = ary > 2
ary[mask] = 1
ary[~mask] = 0 # ~ is bitwise negate opeartion
ary

We can also chain different selection criteria using the logical and operator ‘&’ or the logical or operator ‘|’.

ary = np.array([1, 2, 3, 4])

(ary > 3) | (ary < 2)

The example below demonstrates how we can select array elements that are greater

ary[(ary > 3) & (ary % 2 == 0)]

And, for example, to negate the condition, we can use the ~ operator:

~((ary > 3) | (ary < 2))

Linear algebra with numpy#

numpy by default is using elementwise mulitiplication when you do x*y for two equally sized arrays
What if you wanted to take dot product between arrays as is done in linear algebra between vectors or matrices?
There are special tools to do common linear algebra tasks

\[ \bf{x} \cdot \bf{y} = \sum_i x_i y_i \]

v = np.array([9,10])
w = np.array([11, 12])

# Inner product of vectors; both produce 219
print(v@w) 

x = np.array([[1,2],[3,4]])
v = np.array([9,10])


# Matrix / vector product; both produce the rank 1 array [29 67]
print(x@v) 

x = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])

# Matrix / matrix product; both produce the rank 2 array
print(x@y) 

np.eig(y) # compute eigenvalues and eigenvectors

Aggregation#

Numpy provides many useful functions for performing computations on arrays; one of the most useful is sum:

x = np.array([[1,2],[3,4]])
np.sum(x,axis=1)

print(np.sum(x))   # Compute sum of all elements; prints "10"
print(np.sum(x, axis=0))  # Compute sum of each column; prints "[4 6]"
print(np.sum(x, axis=1))   # Compute sum of each row; prints "[3 7]"

print(x.max())
print(x.min())

Reshaping arrays#

In practice, we often run into situations where existing arrays do not have the right shape to perform certain computations. A
Remember once created that the size of NumPy arrays is fixed
Fortunately, this does not mean that we have to create new arrays and copy values from the old array to the new one if we want arrays of different shapes – the size is fixed, but the shape is not. NumPy provides a reshape methods that allow us to obtain a view of an array with a different shape.

x=np.array([1,2,3,4,5,6,7,8,9,10])

x=x.reshape(2,5)
x

x=x.reshape(5,2)
x

# transpose matrix
x.T 

# add an empty dimension
y = np.arange(3)

print(y.shape)

z = [y: np.newaxis]

print(z.shape)
print(z)

Broadcasting rules of numpy arrays#

Broadcasting is a powerful mechanism that allows numpy to work with arrays of different shapes when performing arithmetic operations. Frequently we have a smaller array and a larger array, and we want to use the smaller array multiple times to perform some operation on the larger array.

The rules of broadcasting are:

Rule 1: If the two arrays differ in their number of dimensions, the shape of the one with fewer dimensions is padded with ones on its leading (left) side.
Rule 2: If the shape of the two arrays does not match in any dimension, the array with shape equal to 1 in that dimension is stretched to match the other shape.
Rule 3: If in any dimension the sizes disagree and neither is equal to 1, an error is raised.

Examples of broadcasting

data     = np.array([[1,2],[3,4],[5,6]])
ones_row = np.array([1,1])
data.shape, ones_row.shape

data

ones_row

data.shape, ones_row.shape

data+ones_row

Let us see both rules in action on another example

a = np.arange(3).reshape((3, 1))
print(a)
print(a.shape)

b = np.arange(3)
print(b)
print(b.shape)

Lets predict a+b sum. By first rule the sum of arrays with shapes (3,1)+(3,) are broadcast to (3,1)+(1,3) then by second rule dimensions one are padded to match the shape (3,3)+(3,3)

a+b

Pandas#

You may thinkg of numpy as enhancing functionality of lists for numerical computations. In the same vein you can think of pandas as enahcnign dicitonaires to deal with heteogenuous categorical data.
Pandas is widely used by data analysts from all disciplines to carry out rapid data cleaning, statistical analysis and plotting. The DataFrame is the ore object of pandas whihc stores observables as columns whith rows indicating measurments or samples. Lets create an example.

import pandas as pd
import numpy as np

Can creatr pandas dataframe from lists, or arrays

A = pd.DataFrame({'Time': [1,2,3,4,5], 
                 'Energy': np.array([10,20,30,40,50])
                 })

A['velocity'] = np.zeros(5)

A.columns

A.index

# acess underlying values as numpy arrays
A['Energy'].values

Problems#

1. Predict and explain the following statements

Create an array of the numbers 1, 5, 19, 30
Create an array of the numbers -3, 15,0.001, 6.02e23
Create an array of integers between -10 and 10
Create an array of 10 equally spaced angles between 0 and \(2\pi\)
Create an array of logarithmically spaced numbers between 1 and 1 million. Hint: remember to pass exponents to the np.logspace() function.
Create an array of 20 random integers between 1 and 10
Create an array of 30 random numbers with a normal distribution
Predict the outcome of the following operation between two NumPy arrays. Test your your prediction.

\[\begin{split} \left[ \begin{array}{cc} 1 & 1 \\ 2 & 2 \end{array} \right] + \left[1 \right] = \,\, ?\end{split}\]
Predict the outcome of the following operation between two NumPy arrays. Test your your prediction.

\[\begin{split} \left[ \begin{array}{ccc} 1 & 8 & 9 \\ 8 & 1 & 9 \\ 1 & 8 & 1 \end{array} \right] + \left[ \begin{array}{cc} 1 & 1 \\ 1 & 1 \end{array} \right] = \,\, ? \end{split}\]
Predict the outcome of the following operation between two NumPy arrays. Test your your prediction.

\[\begin{split} \left[ \begin{array}{cc} 1 & 8 \\ 3 & 2 \end{array} \right] + \left[ \begin{array}{cc} 1 & 1 \\ 1 & 1 \end{array} \right] = \,\, ?\end{split}\]

2. Array Manipulation

Create an array B that contains integers 0 to 24 (including 24) in one row. Then reshape B into a 5 row by 5 column array
Extract the 2nd row from B. Store it as a one column array called x.
Store the number of elements in array x in a new variable called y.
Extract the last column of B and store it in an array called z.
Store a transposed version of B in an array called t.

3. Arrray slicing

The 1D NumPy array G is defined below. But your code should work with any 1D NumPy array filled with numeric values.

G = np.array([5, -4.7, 99, 50, 6, -1, 0, 50, -78, 27, 10])

Select all of the positive numbers in G and store them in x.
Select all the numbers in G between 0 and 30 and store them in y.
Select all of the numbers in G that are either less than -50 or greater than 50 and store them in z.

Generate a one-dimensional array with the following code and index the 5th element of the array.
```
arr = np.random.randint(0, high=10, size=10)
```
Generate a two-dimensional array with the following code.
```
arr2 = np.random.randint(0, high=10, size=15).reshape(5, 3)
```
a. Index the second element of the third column.

b. Slice the array to get the entire third row.

c. Slice the array to access the entire first column.

d. Slice the array to get the last two elements of the first row.

4. random numbers

For the following randomly-generated array:
```
arr = np.random.rand(20)
```
a. Find the index of the largest values in the following array.

b. Calculate the mean value of the array.

c. Calculate the cumulative sum of the array.

d. Sort the array.
Generate a random array of values from -1 \(\rightarrow\) 1 (exclusive) and calculate its median value. Hint: start with an array of values 0 \(\rightarrow\) 1 (exclusive) and manipulate it.
Generate a random array of integers from 0 \(\rightarrow\) 35 (inclusive) and then sort it.
Hydrogen nuclei can have a spin of +1/2 and -1/2 and occur in approximately a 1:1 ratio. Simulate the number of +1/2 hydrogen nuclei in a molecule of six hydrogen atoms and plot the distribution. Hint: being that there are two possible outcomes, this can be simulated using a binomial distribution.

Generating an Combining arrays – Bohr hydrogen atom.

a. Create an array containing the principle quantum numbers (n) for the first eight orbits of a hydrogen atom (e.i., 1 \(\rightarrow\) 8). b. Generate a second array containing the energy (J) of each orbit in part A for a Bohr model of a hydrogen atom using the equation \(E = -2.18 \times 10^{-18}J (\frac{1}{n^2} )\) c. Combine the two arrays from parts A and B into a new 8 \(\times\) 2 array with the first column containing the principle quantum numbers and the second containing the energies. d. Compute transition energies as a function of quantum number separation and make a plot