PyTorch : #1 Tensors

Introduce to Tensors in PyTorch

Tensors are the basic building blocks of data in PyTorch. They are multi-dimensional arrays, similar to NumPy arrays, but with some additional features that make them especially useful for deep learning applications. In this post, we will cover the basics of tensors, including their creation, manipulation, and some important operations that can be performed on them.

What are Tensors?

Tensors are multi-dimensional arrays that can be used to represent data in PyTorch. They are similar to NumPy arrays, but with some additional features that make them especially useful for deep learning applications. Tensors can be created with different data types, such as float, int, and bool, and with different shapes, such as scalars, vectors, matrices, and higher-dimensional arrays.

Tensors in PyTorch

Basic Operations with Tensors

Tensors in PyTorch support all the basic operations that are available with NumPy arrays, such as addition, subtraction, multiplication, and division. Here are some examples:

import torch

# Creating tensors
x = torch.tensor([[1, 2], [3, 4]])
y = torch.tensor([[5, 6], [7, 8]])

# Addition
z = x + y
print(z)  # Output: tensor([[ 6,  8], [10, 12]])

# Subtraction
z = x - y
print(z)  # Output: tensor([[-4, -4], [-4, -4]])

# Multiplication
z = x * y
print(z)  # Output: tensor([[ 5, 12], [21, 32]])

# Division
z = x / y
print(z)  # Output: tensor([[0.2000, 0.3333], [0.4286, 0.5000]])

There are more than 100 tensor operations, including arithmetic, linear algebra, matrix manipulation (transposing, indexing, slicing). For sampling and reviewing, you'll find a comprehensive description here.

Each of these operations can be run on the GPU (at typically higher speeds than on a CPU).

• CPUs have up to 16 cores. Cores are units that do the actual computation. Each core processes tasks in a sequential order (one task at a time).
• GPUs have 1000s of cores. GPU cores handle computations in parallel processing. Tasks are divided and processed across the different cores. That's what makes GPUs faster than CPUs in most cases. GPUs perform better with large data than small data. GPU are typically used for high-intensive computation of graphics or neural networks (we'll learn more about that later in the Neural Network unit).
• PyTorch can use the Nvidia CUDA library to take advantage of their GPU cards.

Operations on Tensor in PyTorch

By default, tensors are created on the CPU. Tensors can also be computed to GPUs; to do that, you need to move them using the .to method (after checking for GPU availability). Keep in mind that copying large tensors across devices can be expensive in terms of time and memory!

Broadcasting

Broadcasting is a useful feature of PyTorch tensors that allows operations to be performed on tensors with different shapes. In broadcasting, the smaller tensor is broadcasted to match the shape of the larger tensor. Here's an example:

import torch

# Creating tensors
x = torch.tensor([[1, 2], [3, 4]])
y = torch.tensor([5, 6])

# Broadcasting
z = x + y
print(z)  # Output: tensor([[ 6,  8], [ 8, 10]])

Slicing and Indexing

Tensors in PyTorch can be sliced and indexed just like NumPy arrays. Here are some examples:

import torch

# Creating a tensor
x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Slicing
y = x[:, 1]
print(y)  # Output: tensor([2, 5, 8])

# Indexing
z = x[1, 2]
print(z)  # Output: tensor(6)

Numpy Interoperability

Tensors in PyTorch can be converted to and from NumPy arrays with ease. Here's an example:

import torch
import numpy as np

# Creating a tensor
x = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Converting to NumPy array
y = x.numpy()
print(y)

# Converting back to tensor
z = torch.from_numpy(y)
print(z)​

 

Initializing a Tensor

Tensor from a Numpy array:

import numpy as np
import torch

a = np.array([1, 2, 3, 4])
b = torch.from_numpy(a)
print(b)

Tensor with a specific shape:

import torch

a = torch.zeros(2, 3)
b = torch.ones(2, 3)
c = torch.rand(2, 3)

print(a)
print(b)
print(c)

Tensor with a specific data type:

import torch

a = torch.zeros(2, 3, dtype=torch.int)
b = torch.ones(2, 3, dtype=torch.float)
c = torch.rand(2, 3, dtype=torch.double)

print(a)
print(b)
print(c)

 

Tensor Attributes

 

import torch

# Initialize a Tensor
a = torch.tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Tensor Shape
print(a.shape)  
# output 
# 	torch.Size([3, 3])

# Tensor Data Type:
print(a.dtype)  
# output 
# 	torch.int64

# Tensor Device:
print(a.device)  
# output 
# 	cpu

# Tensor Number of Dimensions:
print(a.ndim)
# output 
# 	2

 

---

References : 

• TORCH.TENSOR : https://pytorch.org/docs/stable/tensors.html

---