Pytorch | yogeshsn

PyTorch is an open-source machine learning library that is widely used for building and training deep neural networks. It is built on the Python programming language and provides a user-friendly interface for working with tensors, which are multidimensional arrays similar to NumPy arrays but with GPU acceleration.

Some key features of PyTorch include:

Tensors: PyTorch uses tensors as the fundamental data structure for representing data. Tensors can be easily manipulated using various operations like addition, multiplication, and reshaping.
Automatic Differentiation: PyTorch provides an automatic differentiation engine called autograd that automatically computes the gradients of the neural network during training. This makes it easier to train complex models without having to manually calculate the gradients.
Neural Network Building Blocks: PyTorch provides a set of building blocks for constructing neural networks, such as layers, activations, and loss functions. These building blocks can be easily combined to create complex models.
GPU Acceleration: PyTorch supports GPU acceleration, which allows for faster training of models on large datasets.
Dynamic Computation Graphs: PyTorch uses dynamic computation graphs, which means that the structure of the neural network can be modified during runtime. This allows for more flexibility in designing and experimenting with different architectures.

Here’s a simple example of how to create a neural network using PyTorch:

import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(10, 5)
        self.fc2 = nn.Linear(5, 2)
        self.relu = nn.ReLU()

    def forward(self, x):
        x = self.fc1(x)
        x = self.relu(x)
        x = self.fc2(x)
        return x

net = Net()

In this example, we define a simple neural network with two fully connected layers and a ReLU activation function. The forward method defines the forward pass of the network, which takes an input tensor and returns the output tensor.

PyTorch is widely used in various applications such as computer vision, natural language processing, and speech recognition. It is also popular among researchers and developers due to its flexibility and ease of use.

I hope this brief introduction gives you a basic understanding of what PyTorch is and how it can be used for building neural networks. If you want to learn more, there are many online tutorials and resources available that provide more detailed explanations and examples.

Now, let’s move on to PyTorch examples:

First, let’s import the necessary modules:

import torch
import torch.nn as nn
import torch.optim as optim

Creating a Neural Network

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.fc1 = nn.Linear(10, 64)
        self.fc2 = nn.Linear(64, 32)
        self.fc3 = nn.Linear(32, 1)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = torch.relu(self.fc2(x))
        x = torch.sigmoid(self.fc3(x))
        return x

model = Net()

Defining Loss Function and Optimizer

criterion = nn.BCELoss()
optimizer = optim.Adam(model.parameters())

Training Loop

for epoch in range(num_epochs):
    for inputs, labels in dataloader:
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

Making Predictions

with torch.no_grad():
    predictions = model(test_data)

Saving a Model

torch.save(model.state_dict(), 'model.pth')

Loading a Model

model = Net()
model.load_state_dict(torch.load('model.pth'))

Using GPU if Available

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model.to(device)

Data Parallelism
```
model = nn.DataParallel(model)
```

Custom Dataset

class MyDataset(torch.utils.data.Dataset):
    def __init__(self, data, labels):
        self.data = data
        self.labels = labels

    def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        return self.data[idx], self.labels[idx]

DataLoader

dataloader = torch.utils.data.DataLoader(dataset, batch_size=32, shuffle=True)

Transfer Learning

model = torchvision.models.resnet18(pretrained=True)
for param in model.parameters():
    param.requires_grad = False
model.fc = nn.Linear(512, num_classes)

Custom Loss Function

class MyLoss(nn.Module):
    def __init__(self):
        super(MyLoss, self).__init__()

    def forward(self, outputs, labels):
        return torch.mean((outputs - labels)**2)

criterion = MyLoss()

Learning Rate Scheduler

scheduler = optim.lr_scheduler.StepLR(optimizer, step_size=30, gamma=0.1)

Gradient Clipping

torch.nn.utils.clip_grad_norm_(model.parameters(), max_norm=1.0)

Using TensorBoard

from torch.utils.tensorboard import SummaryWriter
writer = SummaryWriter()
writer.add_scalar('Loss/train', loss, epoch)
writer.close()

Citations:

[1] https://omicstutorials.com/introduction-to-pytorch-for-bioinformatics/
[2] https://www.slideshare.net/slideshow/pytorch-introduction/255987453
[3] https://www.youtube.com/watch?v=xWcJjlBXya0
[4] https://www.techtarget.com/searchenterpriseai/definition/PyTorch
[5] https://deeplearning.neuromatch.io/tutorials/W1D1_BasicsAndPytorch/student/W1D1_Tutorial1.html