Data handling - Monthly Flow

Note: The explanation of the code is generated with the assistance of AI Importing Libraries import pandas as pd import matplotlib.pyplot as plt pandas (pd): A powerful Python library for data manipulation and analysis. It provides data structures and functions to efficiently handle structured data, including tabular data such as spreadsheets and SQL tables. matplotlib.pyplot (plt): A Python plotting library that provides a MATLAB-like interface for creating a variety of charts, including line plots, scatter plots, histograms, and more. Loading the CSV File file_path = 'discharge_ai.csv' # Replace with the path to your CSV file df = pd.read_csv(file_path) pd.read_csv(file_path): This function reads a CSV file into a DataFrame. The DataFrame is a two-dimensional labeled data structure with columns of potentially different types. It is similar to an Excel spreadsheet or SQL table, or a dictionary of Series objects. Displaying the First Few Rows of the DataFrame print(df.head()) df.head(): This method returns the first n rows for the object based on position. It is useful for quickly testing if your object has the right type of data in it. By default, it returns the first 5 rows. Converting the ‘Date’ Column to Datetime Format df['Date'] = pd.to_datetime(df['Date']) pd.to_datetime(): This method converts various types of date formats into a standardized datetime format. It is useful for handling date and time data in a consistent manner. Setting the ‘Date’ Column as the Index df.set_index('Date', inplace=True) df.set_index(‘Date’, inplace=True): This method sets the ‘Date’ column as the index of the DataFrame. This allows for efficient data manipulation and analysis based on the date. Checking for Missing Values print(df.isna().sum()) df.isna(): This method returns a boolean DataFrame showing whether each element is missing (NaN) or not. .sum(): This method returns the sum of the boolean values, effectively counting the number of missing values. Handling Missing Values (Forward Fill) df.fillna(method='ffill', inplace=True) df.fillna(): This method fills missing values in the DataFrame. It can be used to fill missing values with a specific value or to forward fill missing values. method=‘ffill’: This method fills missing values by propagating the last valid observation forward. inplace=True: This parameter modifies the DataFrame in place, meaning it changes the original DataFrame. Resampling the Data to Calculate Monthly Averages monthly_avg = df.resample('M').mean() df.resample(‘M’): This method resamples the DataFrame by grouping it by a specified period. In this case, it groups the data by months. .mean(): This method calculates the mean of the resampled data. Displaying the Monthly Averages print(monthly_avg.head()) monthly_avg.head(): This method returns the first n rows for the object based on position. It is useful for quickly testing if your object has the right type of data in it. By default, it returns the first 5 rows. Plotting the Monthly Averages plt.figure(figsize=(10, 6)) plt.plot(monthly_avg.index, monthly_avg['Discharge'], marker='o', linestyle='-') plt.title('Monthly Average River Discharge') plt.xlabel('Date') plt.ylabel('Discharge (cubic meters per second)') plt.grid(True) plt.show() plt.figure(figsize=(10, 6)): This method creates a new figure with the specified size. plt.plot(): This method creates a line plot of the data. plt.title(): This method sets the title of the plot. plt.xlabel() and plt.ylabel(): These methods set the labels for the x and y axes. plt.grid(True): This method adds a grid to the plot. plt.show(): This method displays the plot. Full Script import pandas as pd import matplotlib.pyplot as plt from datetime import datetime # Load the CSV file file_path = 'discharge_ai.csv' df = pd.read_csv(file_path) # Define a function to convert the 'Date' column to datetime format def convert_date(df): df['Date'] = pd.to_datetime(df['Date']) return df # Define a function to set the 'Date' column as the index def set_index(df): df.set_index('Date', inplace=True) return df # Define a function to check for missing values def check_missing(df): print(df.isna().sum()) return df # Define a function to handle missing values def handle_missing(df): df.fillna(method='ffill', inplace=True) return df # Define a function to resample the data to calculate monthly averages def resample_data(df): monthly_avg = df.resample('M').mean() return monthly_avg # Define a function to plot the monthly averages def plot_data(monthly_avg): plt.figure(figsize=(10, 6)) plt.plot(monthly_avg.index, monthly_avg['Discharge'], marker='o', linestyle='-') plt.title('Monthly Average River Discharge') plt.xlabel('Date') plt.ylabel('Discharge (cubic meters per second)') plt.grid(True) plt.show() # Load the CSV file df = pd.read_csv(file_path) # Convert the 'Date' column to datetime format df = convert_date(df) # Set the 'Date' column as the index df = set_index(df) # Check for missing values df = check_missing(df) # Handle missing values df = handle_missing(df) # Resample the data to calculate monthly averages monthly_avg = resample_data(df) # Plot the monthly averages plot_data(monthly_avg) Download CSV File ...

June 2, 2024 · 4 min · Yogesh Sharma Neupane

Understanding Randomness and Hydropower Using Python

In this blog, we will explore how randomness and numpy can be used in a practical example related to water resource engineering and hydropower. We’ll create a program that simulates the operation of a reservoir and calculates hydropower generation. Importing Necessary Libraries import random import numpy as np random: This module helps us generate random numbers. numpy (imported as np): This library is used for numerical operations, especially with arrays and matrices. Defining the Reservoir Class A class in Python is a way to create complex data structures that can hold both data and functions. Here, we define a Reservoir class to model our reservoir. ...

May 31, 2024 · 5 min · Yogesh Sharma Neupane

Python Tutorial: Using random and numpy Libraries

Python Tutorial: Using random and numpy Libraries In this tutorial, we’ll learn how to use the random and numpy libraries in Python. These libraries are powerful tools for generating random numbers and performing numerical operations. We’ll cover everything step-by-step so it’s easy to understand. Importing Libraries import random import numpy as np import random: This imports the random module, which allows us to generate random numbers and make random choices. import numpy as np: This imports the numpy library, a powerful tool for numerical operations, and we use the alias np for convenience. Using the random Module Generating Random Numbers print("Random Integer between 1 and 10:", random.randint(1, 10)) random.randint(1, 10): Generates a random integer between 1 and 10 (inclusive). print("Random float between 0 and 1: {:.3f}".format(random.random())) random.random(): Generates a random float between 0.0 and 1.0. "{:.3f}".format(value): Formats the float to three decimal places. Choosing a Random Element from a List print("Random choice from list:", random.choice([ 'Nepal', 'India', 'China', 'Bhutan', 'Bangladesh', 'Pakistan', 'Afghanistan', 'Iran', 'Iraq', 'Turkey', 'Greece', 'Italy', 'France', 'Spain', 'Portugal', 'United Kingdom', 'Ireland', 'Iceland', 'Canada', 'United States' ])) random.choice(list): Selects a random element from the provided list. Shuffling a List print(f"Random shuffled list: {random.sample(range(8), 7)}") random.sample(range(8), 7): Returns a list of 7 unique elements randomly chosen from the range 0 to 7 (essentially a shuffled list). Setting a Seed for Reproducibility random.seed(42) print("Random Integer with seed 42:", random.randint(1, 10)) random.seed(42): Sets the seed for the random number generator to 42, ensuring that the sequence of random numbers can be reproduced. random.randint(1, 10): Generates a random integer between 1 and 10, which will be the same every time the seed is set to 42. Using numpy for Numerical Operations Generating Random Numbers with numpy random_int_array = np.random.randint(1, 10, size=(3, 3)) print("\nRandom Integer Array:\n", random_int_array) np.random.randint(1, 10, size=(3, 3)): Creates a 3x3 array with random integers between 1 and 9. random_float_array = np.random.rand(3, 3) print("\nRandom Float Array:\n", random_float_array) np.random.rand(3, 3): Creates a 3x3 array with random floats between 0.0 and 1.0. Generating Numbers from a Normal Distribution normal_dist_array = np.random.normal(0, 1, size=(3, 3)) print("\nNormal Distribution Array:\n", normal_dist_array) np.random.normal(0, 1, size=(3, 3)): Creates a 3x3 array with random numbers from a normal (Gaussian) distribution with mean 0 and standard deviation 1. Setting a Seed for Reproducibility in numpy np.random.seed(42) print("\nRandom Float with seed 42:\n", np.random.rand(3, 3)) np.random.seed(42): Sets the seed for NumPy’s random number generator to 42. np.random.rand(3, 3): Creates a 3x3 array with random floats between 0.0 and 1.0, which will be the same every time the seed is set to 42. Array Operations with numpy Basic Array Operations array1 = np.array([1, 2, 3]) array2 = np.array([4, 5, 6]) print("\nArray1:", array1) print("Array2:", array2) np.array([1, 2, 3]): Creates a NumPy array from the list [1, 2, 3]. np.array([4, 5, 6]): Creates a NumPy array from the list [4, 5, 6]. print("Sum of Arrays:", np.add(array1, array2)) np.add(array1, array2): Adds array1 and array2 element-wise. print("Element-wise Multiplication:", np.multiply(array1, array2)) np.multiply(array1, array2): Multiplies array1 and array2 element-wise. print("Dot Product:", np.dot(array1, array2)) np.dot(array1, array2): Computes the dot product of array1 and array2. Statistical Operations data = np.random.rand(1000) print(f"\nData Mean: {np.mean(data):.3f}") print(f"\nData Standard Deviation: {np.std(data):.3f}") print(f"\nData Variance: {np.var(data):.3f}") np.random.rand(1000): Creates an array of 1000 random floats between 0.0 and 1.0. np.mean(data): Calculates the mean (average) of the data. np.std(data): Calculates the standard deviation of the data. np.var(data): Calculates the variance of the data. Reshaping Arrays reshaped_array = np.arange(12).reshape((3, 4)) print("\nReshaped Array:\n", reshaped_array) np.arange(12): Creates an array with values from 0 to 11. reshape((3, 4)): Reshapes the array into a 3x4 array. Slicing Arrays print("\nSliced Array:\n", reshaped_array[:, 1:3]) reshaped_array[:, 1:3]: Slices the array to extract all rows and columns 1 and 2 (excluding column 3). This tutorial covers the basics of using the random and numpy libraries in Python. By following these examples, you’ll gain a solid understanding of how to generate random numbers, perform numerical operations, and manipulate arrays. Happy coding! ...

May 31, 2024 · 4 min · Yogesh Sharma Neupane

Exploring the Use of self in Python with Reservoir Planning

Exploring the Use of self in Python with Reservoir Planning In this blog, we will explore how to use the self function in Python by taking an example from reservoir planning. By the end of this post, we will learn to create a class, define its behavior, and operate it using simple Python code. What is a Class in Python? A class in Python helps us create complex data structures that can contain both data and functions (also called methods) that work on that data. Think of a class as a blueprint for creating objects. Learn more on w3schools ...

May 31, 2024 · 4 min · Yogesh Sharma Neupane

How Matplotlib and Pandas are used? The Basics

In this tutorial, we will be working with streamflow data and using the pandas and matplotlib libraries to analyze and visualize the data. Streamflow Data We will start by generating random streamflow data using the numpy library. We will create an array of 365 random numbers between 5 and 15, representing the streamflow for each day of the year. num_days = 365 streamflow = np.random.uniform(low=5,high=15,size=num_days) Creating a Date Range Next, we will create a date range for the data using the pandas library. We will create a date range from January 26, 2000 to December 31, 2000, with 365 days. ...

May 28, 2024 · 3 min · Yogesh Sharma Neupane

Exploring the Use of self in Python with Reservoir Planning

Exploring the Use of self in Python with Reservoir Planning In this blog, we will explore how to use the self function in Python by taking an example from reservoir planning. By the end of this post, we will learn to create a class, define its behavior, and operate it using simple Python code. What is a Class in Python? A class in Python helps us create complex data structures that can contain both data and functions (also called methods) that work on that data. Think of a class as a blueprint for creating objects. Learn more on w3schools ...

March 31, 2024 · 4 min · Yogesh Sharma Neupane

Introduction to SciPy and Linear Programming

SciPy is an open-source Python library that provides various mathematical algorithms and functions for tasks such as numerical integration, optimization, interpolation, and linear algebra. One of its key modules, scipy.optimize, offers optimization routines including linear programming. Linear programming (LP) is a mathematical method used to determine the best possible outcome, typically defined by an objective function, given a set of linear constraints. It has applications in various fields such as economics, engineering, and operations research. ...

February 10, 2024 · 5 min · Yogesh Sharma Neupane

Headworks Design Day 1

Mr. Bharat Raj Pandey is a senior engineer at Hydro Consult Engineering Limited. Today, he shared valuable lessons in our hydropower engineering class, focusing on hydro headworks design. The initial step involves a topographical survey, requiring a D card from the survey department of Nepal to obtain XYZ coordinates. Control points are established through surveying techniques like triangulation to create a detailed map at a 1:1000 scale. The importance of a detailed map lies in understanding the depth and width needed for foundation work during hydropower headworks construction. Cross-section work, at least 500 meters upstream and downstream, at 20m intervals and 20 meters from the high flood level, is essential for design. Additionally, L-section of the river for about 1 km upstream and downstream of the headworks is crucial. ...

January 14, 2024 · 8 min · Yogesh Sharma Neupane

Grand Ethiopian Renaissance Dam (GERD)

The Grand Ethiopian Renaissance Dam (GERD), formerly known as Millennium Dam and sometimes referred to as Hidese Dam, is a gravity dam situated on the Blue Nile River in the Benishangul-Gumuz Region of Ethiopia. The primary purpose of this dam is to generate electricity, addressing Ethiopia’s energy shortage and facilitating electricity exports to neighboring countries. The initial planned generation capacity of 5250 MW was later increased to 6450 MW with various adjustments made over time. Construction began in 2011 after the eventual site was identified in the 1950s. The main dam height has been raised to 155m, and the reservoir at Full Supply Level (FSL) covers an area of 1875km². Three spillways, with a combined total discharge volume of 38,500 m³/s, are designed for flood events. ...

January 13, 2024 · 9 min · Yogesh Sharma Neupane

Understanding Randomness and Hydropower

In this blog, we will explore how randomness and numpy can be used in a practical example related to water resource engineering and hydropower. We’ll create a program that simulates the operation of a reservoir and calculates hydropower generation. Importing Necessary Libraries import random import numpy as np random: This module helps us generate random numbers. numpy (imported as np): This library is used for numerical operations, especially with arrays and matrices. Defining the Reservoir Class A class in Python is a way to create complex data structures that can hold both data and functions. Here, we define a Reservoir class to model our reservoir. ...

January 6, 2024 · 5 min · Yogesh Sharma Neupane

Exploring Hydrological Methods for Sustainable River Management

Department of Energy Development (DoED) Nepal Guideline In accordance with the Guidelines for the Study of Hydropower Projects 2018, the DoED specifies that a compensation flow (10% of the lowest monthly flow or the amount recommended by IEE/EIA study) should be released downstream. This should be deducted from the available dry season flows for the computation of energy estimates. The government of Nepal plans to harness 10,000 MW by 2026 through various projects. To achieve this goal, it is essential to utilize numerous rivers across Nepal, considering environmental considerations and potential impacts. Determining an appropriate environmental flow figure is a challenge, and studies such as those conducted by Suwal et.al (2020) highlight the need for various indicators of hydrological methods to capture the dynamic nature of river flow in e-flows allocation.Lets discuss some hydrological methods. ...

December 31, 2023 · 3 min · Yogesh Sharma Neupane