Learn Rust by Example: Chia Plotter (3)

Parse (i.e., Regular expression using regex) logs in real-time basis (i.e., real-time file monitoring using Notify) in Rust

This is third part of multi-part tutorial on learning Rust programming language by building an application from scratch, one step (i.e., article) at a time, building upon previous tutorial(s) to finally build a production-ready application.

The following is a list of tutorials for Learn Rust by Example — Chia Plotter:

  1. Execute program or command using std::process::Command in Rust (https://medium.com/geekculture/learn-rust-by-example-chia-plotter-1-29aa268f7cbf)

Previous Tutorial

On the previous tutorial of Learn Rust by Example: Chia Plotter (2), we have developed a stand-alone application that runs three chia plotter processes in parallel, which uses user-configurable parameters. For a full tutorial, please refer to https://medium.com/geekculture/learn-rust-by-example-chia-plotter-2-a574ae09af16.

Tutorial

On this tutorial, we will modify the code to stagger the chia plotting processes based on which phase the chia plotting status is at by parsing logs in real-time. For example, we will

  • Start chia plotting process #1

Tutorial 1: Create a new project then load project in Visual Studio Code

To start, use Cargo to create a new project. Open Windows Powershell and run the following commands:

  1. cd rust

This command generates a new directory named chia-plotter-3 with a src subdirectory and adds two files as shown below.

We will be using Visual Studio Code for all development:

  1. Start Visual Studio Code

5. From the menu, select Run | Run Without Debugging. When prompted with a message box stating “Cargo.toml has been detected in this workspace. Would you like to generate launch configurations for its target?”, select “Yes”. This is a key step for you to run rust programs from Visual Studio Code.

6. From the menu, select Run | Run Without Debugging again to see the following output

Tutorial 2: Parse log file

We will be using regex — Rust (docs.rs) to parse log files to derive status of Chia Plotter process from the log.

Step 1: Analyze Chia Plotter Log

Before we start, we will analyze chia plotting process and its log. Chia plotting process consists of four phases where:

  • Phase 1: Computes table 1 thru table 7 in temporary directory

A snippet from the Chia Plotter log is shown below.

At end of each phase, it writes phase completion duration to the log using syntax “Time for phase #: ####.### seconds.” where # represents a number. An example would be “Time for phase 1 = 9498.982 seconds.” For this tutorial, we will use regex to look for “Time for Phase 1” in the log.

Step 2: Change cargo.toml to include regex

Regex is third-party crate. Whenever we want to add dependent crates to our project, we can rely on Cargo to do all the heavy lifting for us. To depend on a library hosted on crates.io, add it to your Cargo.toml file as shown below. For more information, please refer to Add third-party crates to your project — Learn | Microsoft Docs.

Regex is a library for parsing, compiling, and executing regular expressions. Its syntax is like Perl-style regular expressions but lacks a few features like look around and backreferences. In exchange, all searches execute in linear time with respect to the size of the regular expression and search text.

To use regex, you will need to include the following to your cargo.toml file as shown below. This crate is on crates.io and can be used by adding regex to your dependencies in your project’s Cargo.toml.

[dependencies]
regex = "1"

Step 3: Develop your code to use regex to parse log file

After you have added regex to cargo.toml file, switch to main.rs to start developing your code using regex.

The following is full extract of the code using regex as shown below.

To use regex, you would need include the following lines in your code:

  • use regex::Regex;

The following Rust standard libraries are included for file operation — i.e., log file:

  • use std::fs::File;

Function read_lines accept plotter log filename as input then returns reads line of the file at a time.

Function main set a regular expression we are looking for then look for a match. If match is found then it returns regular expression we are looking for, otherwise returns blank.

Finally, from the menu, select Run | Run Debugging again to see the following output, which prints a series of blank lines and one line of match found as shown below.

Tutorial 3: Real-time log file monitoring using Notify

We will be using https://github.com/notify-rs/notify to monitor log files in real-time basis for any changes.

Step 1: Analyze Chia Plotting Process

Chia plotting is the process of creating plot files that are later harvested as part of the Chia farming process. Chia plotting process takes between six to twelve hours where it goes through four phases. In general, the breakdown of duration of each phase are as follows:

  • Phase 1: 45%

Since different phase of Chia plotting impacts various parts of computer hardware, such as IO, memory, or CPU differently; we would like to avoid parallel Chia plotting tasks as much as possible to avoid over taxing of our computer. For example, if we over tax memory usage then computer operating system enlarges swap file, negatively impacting IO performance and so for.

To avoid over taxing our computer, we would like an automated process of staggering Chia plotting process. We accomplish this by deriving Chia plotting status by monitoring log files.

Step 2: Change cargo.toml to include notify

Notify is third-party crate. Whenever we want to add dependent crates to our project, we can rely on Cargo to do all the heavy lifting for us. To depend on a library hosted on crates.io, add it to your Cargo.toml file as shown below. For more information, please refer to Add third-party crates to your project — Learn | Microsoft Docs.

Notify is cross-platform filesystem notification library for Rust.

To use notify, you will need to include the following to your cargo.toml file as shown below. This crate is on crates.io and can be used by adding notify to your dependencies in your project’s Cargo.toml.

[dependencies]crossbeam-channel = "0.4.0"notify = "5.0.0-pre.7"

Step 3: Develop your code to use notify to look for changes to file(s) in a directory

After you have added regex to cargo.toml file, switch to main.rs to start developing your code using regex.

The following is full extract of the code using notify as shown below.

To use notify, you would need include the following lines in your code:

  • use notify::{RecommendedWatcher, RecursiveMode, Watcher};

The following Rust standard libraries are included for file operation — i.e., log file:

  • use std::path::Path;

Function watch monitors for changes to Chia plotter log files then prints if the file has changed.

Function main set a path where we are monitoring for changes.

Finally, from the menu, select Run | Run Debugging again to see the following output, which prints a message stating that it is watching for changes and prints which file has changed as shown below.

Tutorial 4: Combine regex and real-time log file monitoring

We will be combining both real-time log file monitoring with regex where whenever the log file changes, it parses the log file for a key phrase and prints out that phrase.

The following changes were made to the code:

  • Merged the regex code with notify code by copying regex’s function read_lines() and copying and renaming regex’s function main() to function parse_keystring().

Tutorial 5: Merge the code from previous tutorial with the code from this tutorial

Finally merge the code from previous tutorial with the code from this tutorial to complete the Chia Plotter application as shown below.

[This section needs to be completed.]

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store