A high availability (HA) architecture using Keepalived and Nginx

Here’s an example configuration for setting up a high availability (HA) architecture using Keepalived and Nginx:

  1. Install Keepalived and Nginx on each server.
  2. Configure Keepalived on both servers:
    • Edit the Keepalived configuration file (e.g., /etc/keepalived/keepalived.conf) on each server.
    • Specify the virtual IP (VIP) address that will be used for HA. For example, set virtual_ipaddress to “192.168.0.100”.
    • Configure the VRRP instance with VRRP authentication and priority settings.
    • Define the health check script to monitor the availability of the main server.
  3. Configure Nginx on each server:
    • Edit the Nginx configuration file (e.g., /etc/nginx/nginx.conf) on each server.
    • Define the upstream servers where the traffic will be load balanced. For example, set upstream backend with the IP addresses and ports of your application servers.
    • Configure the server block to listen on the VIP address and proxy the requests to the upstream servers.
  4. Start and enable Keepalived and Nginx services on both servers.

Once you have configured Keepalived and Nginx on both servers, the primary server will hold the VIP address and handle the incoming traffic. If the primary server becomes unavailable, Keepalived will automatically transfer the VIP to the secondary server, which will take over the traffic handling using Nginx.

Please note that this is a simplified example, and the actual configuration may vary based on your specific requirements and environment. It’s important to refer to the documentation and guides provided by Keepalived and Nginx to ensure a proper configuration for your HA setup.

Here’s a simplified example of a Keepalived configuration file (/etc/keepalived/keepalived.conf):

angelscript

vrrp_instance VI_1 {
    state MASTER  # Set this to BACKUP on the secondary server
    interface eth0  # Specify the network interface to monitor
    virtual_router_id 51  # An arbitrary ID for the VRRP instance
    priority 100  # Set this to a lower value (e.g., 90) on the secondary server

    authentication {
        auth_type PASS
        auth_pass your_password  # Set a strong password for VRRP authentication
    }

    virtual_ipaddress {
        192.168.0.100  # The virtual IP address to be used for HA
    }

    track_script {
        chk_nginx  # Name of the health check script for monitoring Nginx
    }
}

In this example, the primary server is configured as the MASTER with a higher priority, while the secondary server should be configured as BACKUP with a lower priority. The interface parameter specifies the network interface to monitor for connectivity. The virtual_router_id is an arbitrary ID used to identify the VRRP instance.

The authentication section defines the authentication method and password for VRRP. It’s important to set a strong password to ensure secure communication between the servers.

The virtual_ipaddress block specifies the virtual IP address (VIP) to be used for the HA setup. In this case, the VIP is set to 192.168.0.100.

The track_script section allows you to define the health check script (e.g., chk_nginx) that monitors the availability of Nginx.

Please note that this is a simplified example, and you may need to adjust the configuration based on your specific setup, network interface, authentication requirements, and health check script. Make sure to consult the Keepalived documentation and customize the configuration file accordingly.

Here’s a more comprehensive example of a Keepalived configuration file (/etc/keepalived/keepalived.conf) that includes additional directives:

plaintext

! Configuration example for Keepalived

global_defs {
    router_id HA_Cluster  # An arbitrary unique identifier for the router
}

vrrp_script chk_nginx {
    script "/usr/local/bin/check_nginx.sh"  # Path to the custom health check script
    interval 10  # Check interval in seconds
    timeout 3  # Timeout in seconds
    fall 3  # Number of consecutive failures to consider a server as down
    rise 2  # Number of consecutive successes to consider a server as up
}

vrrp_instance VI_1 {
    state MASTER  # Set this to BACKUP on the secondary server
    interface eth0  # Specify the network interface to monitor
    virtual_router_id 51  # An arbitrary ID for the VRRP instance
    priority 100  # Set this to a lower value (e.g., 90) on the secondary server

    authentication {
        auth_type PASS
        auth_pass your_password  # Set a strong password for VRRP authentication
    }

    virtual_ipaddress {
        192.168.0.100  # The virtual IP address to be used for HA
    }

    track_script {
        chk_nginx  # Name of the health check script for monitoring Nginx
    }

    notify_master "/usr/local/bin/master.sh"  # Path to the custom script to execute when becoming the master
    notify_backup "/usr/local/bin/backup.sh"  # Path to the custom script to execute when becoming the backup
    notify_fault "/usr/local/bin/fault.sh"  # Path to the custom script to execute on a fault event

    smtp_alert  # Enable email alerts in case of state transition events
    smtp_server your_smtp_server  # SMTP server address for email alerts
    smtp_connect_timeout 30  # SMTP connection timeout in seconds
    smtp_interface eth0  # Specify the network interface for email alerts
    smtp_from your_email@your_domain.com  # Sender email address for email alerts
    smtp_to your_alert_email@your_domain.com  # Recipient email address for email alerts
    smtp_subject "Keepalived Notification"  # Subject line for email alerts
}

In this example, the global_defs section sets a unique identifier for the router.

The vrrp_script section defines a health check script (chk_nginx) that monitors the availability of Nginx. You can replace /usr/local/bin/check_nginx.sh with the path to your custom health check script. The intervaltimeoutfall, and rise parameters define the check interval, timeout, and criteria for determining the server’s state.

The vrrp_instance section configures the VRRP instance. The state parameter should be set to MASTER on the primary server and BACKUP on the secondary server. The interface parameter specifies the network interface to monitor. The virtual_router_id is an arbitrary ID used to identify the VRRP instance. The priority parameter determines the priority of the server, with a higher value indicating a higher priority.

The authentication section sets the authentication method and password for VRRP communication.

The virtual_ipaddress block specifies the virtual IP address (VIP) to be used for the HA setup.

The track_script section references the health check script to monitor Nginx availability.

The notify_masternotify_backup, and notify_fault directives specify custom scripts to execute on state transition events.

The smtp_alert directive enables email alerts for state transition events. You can configure the SMTP server and email addresses in the subsequent lines.

Please note that this configuration is provided as an example, and you will need to customize it according to your specific setup, file paths, authentication requirements, and notification scripts. Refer to the Keepalived documentation for further details on each directive and adapt the configuration to meet your needs.

 

Here’s an example of a Keepalived configuration file (/etc/keepalived/keepalived.conf) for the secondary server configured as BACKUP:

plaintext

! Configuration example for Keepalived (BACKUP)

global_defs {
    router_id HA_Cluster  # An arbitrary unique identifier for the router
}

vrrp_script chk_nginx {
    script "/usr/local/bin/check_nginx.sh"  # Path to the custom health check script
    interval 10  # Check interval in seconds
    timeout 3  # Timeout in seconds
    fall 3  # Number of consecutive failures to consider a server as down
    rise 2  # Number of consecutive successes to consider a server as up
}

vrrp_instance VI_1 {
    state BACKUP  # Set this to MASTER on the primary server
    interface eth0  # Specify the network interface to monitor
    virtual_router_id 51  # An arbitrary ID for the VRRP instance
    priority 90  # Set this to a higher value (e.g., 100) on the primary server

    authentication {
        auth_type PASS
        auth_pass your_password  # Set the same strong password used on the primary server
    }

    virtual_ipaddress {
        192.168.0.100  # The virtual IP address to be used for HA
    }

    track_script {
        chk_nginx  # Name of the health check script for monitoring Nginx
    }

    notify_master "/usr/local/bin/master.sh"  # Path to the custom script to execute when becoming the master
    notify_backup "/usr/local/bin/backup.sh"  # Path to the custom script to execute when becoming the backup
    notify_fault "/usr/local/bin/fault.sh"  # Path to the custom script to execute on a fault event

    smtp_alert  # Enable email alerts in case of state transition events
    smtp_server your_smtp_server  # SMTP server address for email alerts
    smtp_connect_timeout 30  # SMTP connection timeout in seconds
    smtp_interface eth0  # Specify the network interface for email alerts
    smtp_from your_email@your_domain.com  # Sender email address for email alerts
    smtp_to your_alert_email@your_domain.com  # Recipient email address for email alerts
    smtp_subject "Keepalived Notification"  # Subject line for email alerts
}

The /usr/local/bin/master.sh script mentioned in the Keepalived configuration file is a custom script that you can create and define according to your specific requirements when the server becomes the master in the high availability setup. This script allows you to perform any necessary actions or configurations when the server transitions to the master state.

Here’s an example of a simple /usr/local/bin/master.sh script:

bash

#!/bin/bash

# This script is executed when the server becomes the master in the high availability setup

# Add your custom actions or configurations here
echo "Server is now the master. Performing custom actions..."
# Example: Start additional services or update configurations

# Restart Nginx to ensure it's using the VIP
systemctl restart nginx

In this example, the script starts by printing a message indicating that the server is now the master and then proceeds to perform any necessary custom actions or configurations. You can add your own logic to the script, such as starting additional services, updating configurations, or performing any other tasks required when the server becomes the master.

In this specific example, the script restarts Nginx to ensure that it is using the virtual IP (VIP) address, which is now assigned to the master server.

Remember to make the script executable by running the following command:

bash

chmod +x /usr/local/bin/master.sh

You can modify the /usr/local/bin/master.sh script according to your specific needs and include any additional commands or configurations that are relevant to your high availability setup.

 

The /usr/local/bin/check_nginx.sh script mentioned in the Keepalived configuration file is a custom health check script that monitors the availability of Nginx. This script is executed periodically by Keepalived to determine the state of the server and make decisions based on the health check results.

Here’s an example of a simple /usr/local/bin/check_nginx.sh script:

bash

#!/bin/bash

# This script checks the availability of Nginx

# Perform a health check on Nginx
response=$(curl -s -o /dev/null -w "%{http_code}" http://localhost)

# Check the HTTP response code
if [ "$response" = "200" ]; then
    exit 0  # Nginx is healthy, exit with 0 (success)
else
    exit 1  # Nginx is down or not responding properly, exit with 1 (failure)
fi

In this example, the script uses the curl command to send a request to http://localhost and retrieves the HTTP response code. If the response code is 200 (indicating a successful response), the script exits with a status of 0, indicating that Nginx is healthy. Otherwise, if the response code is different from 200, the script exits with a status of 1, indicating that Nginx is down or not responding properly.

You can customize the health check script according to your specific requirements and the way you want to check the health of Nginx. For example, you may want to check additional conditions, such as the response time, specific content on the page, or perform more complex checks. Adapt the script to suit your needs and ensure that it exits with a status of 0 (success) when Nginx is healthy and 1 (failure) when Nginx is down or not responding as expected.

Don’t forget to make the script executable by running the following command:

bash

chmod +x /usr/local/bin/check_nginx.sh

This allows Keepalived to execute the script and perform the health checks at the specified interval in the configuration file.

 

 

The /usr/local/bin/backup.sh script mentioned in the Keepalived configuration file is a custom script that you can create and define according to your specific requirements when the server transitions to the backup state in the high availability setup. This script allows you to perform any necessary actions or configurations on the backup server.

Here’s an example of a simple /usr/local/bin/backup.sh script:

bash

#!/bin/bash

# This script is executed when the server becomes the backup in the high availability setup

# Add your custom actions or configurations here
echo "Server is now the backup. Performing custom actions..."
# Example: Stop unnecessary services or disable certain functionalities

# Stop Nginx to prevent conflicts with the master server
systemctl stop nginx

In this example, the script starts by printing a message indicating that the server is now the backup and then proceeds to perform any necessary custom actions or configurations. You can add your own logic to the script, such as stopping unnecessary services, disabling certain functionalities, or performing any other tasks required when the server transitions to the backup state.

In this specific example, the script stops Nginx to prevent conflicts with the master server since the backup server should not be actively serving traffic while in the backup state.

Remember to make the script executable by running the following command:

bash

chmod +x /usr/local/bin/backup.sh

You can modify the /usr/local/bin/backup.sh script according to your specific needs and include any additional commands or configurations that are relevant to your high availability setup.

 

 

The /usr/local/bin/fault.sh script mentioned in the Keepalived configuration file is a custom script that you can create and define according to your specific requirements to handle fault events in the high availability setup. This script is executed when a fault event occurs, indicating a problem with the server or the high availability configuration.

Here’s an example of a simple /usr/local/bin/fault.sh script:

bash

#!/bin/bash

# This script is executed on a fault event in the high availability setup

# Add your custom actions or configurations here
echo "Fault event detected. Performing custom actions..."
# Example: Send notifications, log the event, or trigger failover procedures

# Restart Keepalived to initiate failover
systemctl restart keepalived

In this example, the script starts by printing a message indicating that a fault event has been detected and then proceeds to perform any necessary custom actions or configurations. You can add your own logic to the script, such as sending notifications, logging the event, triggering failover procedures, or performing any other tasks required when a fault event occurs.

In this specific example, the script restarts Keepalived to initiate a failover procedure. Restarting Keepalived can help recover from certain types of faults and trigger the transition to a new master server if necessary.

Remember to make the script executable by running the following command:

bash

chmod +x /usr/local/bin/fault.sh

You can modify the /usr/local/bin/fault.sh script according to your specific needs and include any additional commands or configurations that are relevant to handling fault events in your high availability setup.

MySQL Master Slave Replication: 7 Easy Steps

1. Setting Up The Master
The first thing you need to accomplish in the MySQL master-slave replication process is to install and configure the master server. If you have not installed MySQL, then you can install MySQL using the following command:

root@repl-master:~# sudo apt-get update
root@repl-master:~# sudo apt-get install mysql-server mysql-client -y
root@repl-master:~# sudo mysql_secure_installation
Read through Installing MySQL on Ubuntu 20.04: 6 Easy Steps for more insights.

Once the MySQL installation process is completed, use the following command to edit the MySQL configuration file:

root@repl-master:~# sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf
Next, in the same file, find the line containing bind-address = 127.0.0.1 and replace that IP address with the IP address of your master replication server. So, the line will look like:
bind-address = 12.34.56.111

Next, find the following lines in the file:

server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
You will see that the above lines have been commented, just uncomment these lines and exit the edit interface by clicking CTRL + X. Save the changes and restart the MySQL service for the changes to take effect.

Restart MySQL service using the following command:

root@repl-master:~# sudo service mysql restart
2. Create A New User For Slave
The next step is to create a new user for your slave server. Use the following command to create it:

root@repl-master:~# mysql -uroot -p;
mysql> CREATE USER ‘slave’@’12.34.56.789‘ IDENTIFIED BY ‘SLAVE_PASSWORD‘;
mysql> GRANT REPLICATION SLAVE ON . TO ‘slave’@’12.34.56.222 ‘;
mysql> FLUSH PRIVILEGES;
mysql> FLUSH TABLES WITH READ LOCK;
You will use the following command to know the current status of the master server:

mysql> SHOW MASTER STATUS;
This command will also tell the slave to follow the master from this position.

3. Move Data From Master To Slave
Now that you have marked the position, you can start moving the data from the master to the slave. You need to create a MySQL dump file to move the data. Use the following command to create the dump file:

root@repl-master:~# mysqldump -u root -p –all-databases –master-data > data.sql
To copy the dump file to the slave, use the following command:

scp data.sql root@12.34.56.222
Unlock the tables using the following command:

mysql> UNLOCK TABLES;
4. Configure Slave Server
Now, all you need to do is configure the slave server and test if replication is working. Ensure MySQL is installed.
Open the configuration file in your slave server and update these lines:

root@repl-slave:~# sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf
In the same way that you did for the master server, you need to bind the IP address and uncomment those two lines for the slave server.
Now, restart the MySQL server using the following command:

root@repl-slave:~# sudo service mysql restart
5. Import Data Dump
Use the following command to import the dump file to the slave server:

root@repl-slave:~# mysql -uroot -p < data.sql Once the data is imported, you need to stop MySQL in the slave server using the following command: root@repl-slave:~# mysql -uroot -p; mysql> STOP SLAVE;
You have finally imported the dump files and updated the master IP address, password, log file name, and position, to enable the master to communicate with the slave without any issues.

6. Start Slave Server
Next, use the “Start Slave” command to start operating the slave server.

START SLAVE;
7. Test MySQL Master Slave Replication
To test if your MySQL master slave replication works, just create a database in your master server and see if it is replicated in the slave server. If you can see the database in the slave, then it is working fine.

Create a test database in a master server called ‘sampledb’.

CREATE DATABASE sampledb;
Now login to your slave server and list the databases, and if you see the “sampledb” there, then the master slave replication process is working fine.

Login to your slave server and use the following command to list all databases:

show databases;

Collect and visualize MySQL server logs with the updated MySQL integration for Grafana Cloud

Today, we are excited to announce that the MySQL integration has received an important update, which includes a new pre-built MySQL logs dashboard and the Grafana Agent configuration to view and collect MySQL server logs.

The integration is already available in Grafana Cloud, our platform that brings together all your metrics, logs, and traces with Grafana for full-stack observability.

Why you need logs

Of all the three pillars of observability, metrics are the most widely used: They are easier to gather and store than logs or traces. They are great for detecting problems and understanding system performance at a glance. Still, metrics are often not enough to understand what caused an issue.

On the other hand, logs can tell you many more details about the root cause, once you narrow down the time and location of the problem using metrics.

Getting started with the MySQL integration

Grafana Agent is the universal collector and is all you need to send different telemetry data to the Grafana Cloud stack, including metrics, logs, and traces.

If you already use the embedded Agent integration to collect Prometheus metrics, your Agent configuration could look like this:

yaml

integrations:
  prometheus_remote_write:
    - url: https://<cloud-endpoint>/api/prom/push
  mysqld_exporter:
    enabled: true
    instance: mysql-01
    data_source_name: "root:put-password-here@(localhost:3306)/"

Adding MySQL logs is just adding some extra lines of Grafana Agent config.yml:

yaml

metrics:
  wal_directory: /tmp/wal
logs:
  configs:
  - name: agent
    clients:
    - url: https://<cloud-logs-endpoint>/loki/api/v1/push
    positions:
      filename: /tmp/positions.yaml
    target_config:
      sync_period: 10s
    scrape_configs:
    - job_name: integrations/mysql 
      static_configs:
        - labels:
            instance: mysql-01
            job: integrations/mysql
            __path__: /var/log/mysql/*.log
      pipeline_stages:
        - regex:
            expression: '(?P<timestamp>.+) (?P<thread>[\d]+) \[(?P<label>.+?)\]( \[(?P<err_code>.+?)\] \[(?P<subsystem>.+?)\])? (?P<msg>.+)'
        - labels:
            label:
            err_code:
            subsystem:
        - drop:
            expression: "^ *$"
            drop_counter_reason: "drop empty lines"

integrations:
  prometheus_remote_write:
    - url: https://<cloud-endpoint>/api/prom/push
  mysqld_exporter:
    enabled: true
    instance: mysql-01
    data_source_name: "root:put-password-here@(localhost:3306)/"
    relabel_configs:
      - source_labels: [__address__]
        target_label: job
        replacement: 'integrations/mysql'

The additional configuration above locates and parses MySQL server logs by using an embedded Promtail Agent.

The most crucial configuration part is to make sure that the labels job and instance match each other for logs and metrics. This ensures that we can quickly dive from graphs to corresponding logs for more details on what actually happened.

You can find more information on configuring the MySQL integration in our MySQL integration documentation.

To learn more and get a better understanding of how to correlate metrics, logs, and traces in Grafana, I also recommend checking out the detailed talk by Andrej Ocenas on how to successfully correlate metrics, logs, and traces in Grafana.

Start monitoring with the MySQL logs dashboard

New logs dashboard in the My SQL integration for Grafana Cloud
New logs dashboard in the My SQL integration for Grafana Cloud

Along with coming packaged with pre-built dashboards as well as metrics and alerts, the MySQL integration for Grafana Cloud now bundles a new MySQL logs dashboard that can be quickly accessed from the MySQL overview dashboard when you need a deeper understanding of what’s going on with your MySQL server:

The important thing to note is that if you jump from one dashboard to another, the context of the MySQL instance and time interval will remain the same.

Try out the MySQL integration

The enhanced MySQL integration with log capabilities is available now for Grafana Cloud users. If you’re not already using Grafana Cloud, we have a generous free forever tier and plans for every use case. Sign up for free now!

It’s the easiest way to get started observing metrics, logs, traces, and dashboards.

For more information on monitoring and alerting on Grafana Cloud and MySQL, check out our MySQL integration documentation,  the MySQL solutions page, or join the #integrations channel in the Grafana Labs Community Slack.

關於Python 3虛擬環境(venv)

Python是一種流行的高級編程語言,廣泛用於Web開發、數據科學、人工智能等領域。為了保持Python的穩定性和可靠性,Python 3引入了venv(虛擬環境)的概念,這使得開發人員可以為每個Python應用程序建立獨立的環境,這些環境之間相互獨立,不會干擾彼此。

venv是Python 3的內置模塊,可以通過命令行創建和管理虛擬環境。下面我們將一步步介紹如何使用venv創建虛擬環境。

首先,打開命令行終端,進入你想要創建虛擬環境的目錄。然後運行以下命令:

“`
python3 -m venv myenv
“`

這個命令將在當前目錄中創建一個名為“myenv”的虛擬環境。你可以將“myenv”替換為你想要的任何名稱。

接下來,啟動虛擬環境。在命令行中運行以下命令:

– 在Windows中:

“`
myenv\Scripts\activate.bat
“`

– 在Mac或Linux中:

“`
source myenv/bin/activate
“`

這會激活虛擬環境,你將看到命令行前面出現了虛擬環境的名稱。

現在你可以安裝你需要的Python庫或模塊。在虛擬環境中運行以下命令:

“`
pip install <package_name>
“`

當你完成開發工作後,你可以退出虛擬環境。在命令行中運行以下命令:

“`
deactivate
“`

這將退出虛擬環境,你將回到原來的Python環境。

總之,venv是一個非常有用的工具,它可以幫助開發人員管理Python應用程序的依賴性並避免版本衝突。如果你還沒有使用venv,那麼現在是時候開始使用它了!

ESP32 vs ESP8266 – Pros and Cons

What’s the difference between ESP32 and ESP8266? Should you use the ESP32 or the ESP8266 in your projects? In this article, we’ll compare the ESP32 with the ESP8266 and cover the pros and cons of each board.

The ESP32 and ESP8266 are cheap Wi-Fi modules perfectly suited for DIY projects in the Internet of Things (IoT) and Home Automation fields.

Both chips have a 32-bit processor. The ESP32 is a dual-core 160MHz to 240MHz CPU, whereas the ESP8266 is a single-core processor that runs at 80MHz.

These modules come with GPIOs that support various protocols like SPI, I2C, UART, ADC, DAC, and PWM. The best part is that these boards come with wireless networking included, which makes them apart from other microcontrollers like the Arduino. This means that you can easily control and monitor devices remotely via Wi-Fi or Bluetooth (in the case of ESP32) for a very low price.

Alternatively, if you don’t need to use its wireless capabilities, you can use the ESP32/ESP8266 to control inputs and outputs as you would do with an Arduino. However, you should take into account that whereas the Arduino works with 5V logic, the ESP32 and ESP8266 work at 3.3V.

Specifications: ESP32 vs ESP8266

The ESP32 is the ESP8266 successor. It adds an extra CPU core, faster Wi-Fi, more GPIOs, and supports Bluetooth 4.2 and Bluetooth low energy. Additionally, the ESP32 comes with touch-sensitive pins that can be used to wake up the ESP32 from deep sleep, a built-in hall effect sensor, and a built-in temperature sensor (recent versions of the ESP32 don’t come with a built-in temperature sensor anymore).

Both boards are cheap, but the ESP32 costs slightly more. While the ESP32 can cost around $6 to $12, the ESP8266 can cost $4 to $6 (but it really depends on where you get them and what model you’re buying).

The following table shows the main differences between the ESP8266 and the ESP32 chips (table adapted from AMICA_IO).

ESP8266
ESP32
MCU
Xtensa Single-core 32-bit L106
Xtensa Dual-Core 32-bit LX6 with 600 DMIPS
802.11 b/g/n Wi-Fi
HT20
HT40
Bluetooth
X
Bluetooth 4.2 and BLE
Typical Frequency
80 MHz
160 MHz
SRAM
X
Flash
X
GPIO
17
34
Hardware /Software PWM
None / 8 channels
None / 16 channels
SPI/I2C/I2S/UART
2/1/2/2
4/2/2/2
ADC
10-bit
12-bit
CAN
X
Ethernet MAC Interface
X
Touch Sensor
X
Temperature Sensor
X
(old versions)
Hall effect sensor
X
Working Temperature
-40ºC to 125ºC
-40ºC to 125ºC
Price
$ (3$ – $6)
$$ ($6 – $12)
Where to buy

Using ESP32 or ESP8266 bare chips is not easy or practical, especially when testing and prototyping. Most of the time, you’ll want to use ESP32 and ESP8266 development boards. These boards come with all the needed circuitry to power the chip, connect it to your computer, a circuit to upload code easily, pins to connect peripherals, built-in power and control LEDs, and other useful features.

The ESP32 and ESP8266 development boards we use more often are the ESP32 DEVKIT DOIT Development board and the ESP8266 ESP-12E NodeMCU Kit and these are the ones we recommend for beginners. However, there are many other models of development boards that you can choose from. We recommend that you read the following guides:

More GPIOs on the ESP32

The ESP32 has more GPIOs than the ESP8266, and you can decide which pins are UART, I2CSPI—you need to set that on the code. This is possible due to the ESP32 chip’s multiplexing feature that allows you to assign multiple functions to the same pin.

If you don’t set them on the code, they will be on the pins defined by default, as shown in the following figure (this is an example for the ESP32 DEVKIT V1 DOIT board (version with 36 GPIOS)—the pin location can change depending on the manufacturer).

To learn more about the ESP32 GPIOs and how to use them, read:

For means of comparison, here’s the pinout diagram for the ESP8266 ESP-12E NodeMCU Kit.

To learn more about the ESP8266 GPIOs and how to use them, read:

PWM, ADC, and More

You can set PWM signals in any GPIO with configurable frequencies and duty cycle set on the code.

When it comes to the analog pins, these are static, but the ESP32 supports measurements on 18 channels (analog-enabled pins) versus just one 10-bit ADC pin on the ESP8266. The ESP32 also supports two 8-bit DAC channels.

Moreover, the ESP32 contains 10 capacitive sensing GPIOs, that detect touch and can be used to trigger events, or wake-up the ESP32 from deep sleep, for example.

The ESP32 supports Bluetooth communication protocol by default, while the ESP8266 doesn’t.


Arduino IDE – ESP32 vs ESP8266

There are many ways to program the ESP32 and ESP8266 boards. Both boards can be programmed with the Arduino core using the Arduino IDE or other IDEs (like VS Code with the PlatformIO extension).

These are good news, especially for those used to program the Arduino board and are familiar with the Arduino “programming language”.

Getting started with the ESP32 or ESP8266 using Arduino IDE and have your first project running is very simple. You can follow these guides:

Although you can program both boards using Arduino IDE, they might not be compatible with the same libraries and functions. Some libraries are just compatible with one of the boards. This means that most of the time, your ESP8266 code will not be compatible with the ESP32. However, usually, you need to make a few modifications.

We have a dedicated list of free tutorials and projects for the ESP32 and ESP8266 boards using the Arduino IDE that you might found useful:

MicroPython Firwmare – ESP32 vs ESP8266

Another popular way of programming the ESP32 and ESP8266 boards is using MicroPython firmware.

MicroPython is a re-implementation of Python 3 targeted for microcontrollers and embedded systems. MicroPython is very similar with regular Python. So, if you already know how to program in Python, you also know how to program in MicroPython.

In MicroPython, most Python scripts are compatible with both boards (unlike when using Arduino IDE). This means that most of the time, you can use the same script for ESP32 and ESP8266.

You can get started with MicroPython firmware on the ESP32 and ESP8266 very quickly by following our free guides:

We also have a list of free projects using MicroPython with the ESP32 and ESP8266 boards:

Need Resources to Get Started?

If you want to get started with the ESP32 or ESP8266, you can take a look at our courses and projects:

ESP32 or ESP8266?

So, at this point you may be wondering: Should I get an ESP8266 or an ESP32?

It really depends on what you want to do. There is space for both boards, and both have pros and cons.

The ESP8266 is cheaper than the ESP32. Although it doesn’t have as many functionalities, it works just fine for most simple DIY IoT projects. However, it has some limitations in the GPIO mapping, and it might not have enough pins for what you intend to do. If that’s the case, you should get an ESP32.

The ESP32 is much more powerful than the ESP8266, comes with more GPIOs with multiple functions, faster Wi-Fi, and supports Bluetooth. However, many people think that the ESP32 is more difficult to deal with than the ESP8266 because it is more complex. On the contrary, in our opinion, it is as easy to program the ESP32 as the ESP8266, especially if you intend to program it using the “Arduino language” or MicroPython.

The ESP32 has some cons too. The ESP32 is more expensive than the ESP8266. So, if you’re building a simple IoT project, the ESP8266 might do the trick for a lower price. Additionally, because the ESP8266 is “older” than the ESP32, some libraries and features are better developed for the ESP8266, and you’ll find more resources (forums, people with the same issues, and how to solve them, etc.). However, as time goes by, the ESP32 is being widely adopted, and these differences in terms of development and libraries won’t be noticeable.

My personal experience: in 2021, I use almost exclusively the ESP32 for IoT projects. It is more versatile, and it comes with much more functionalities like Bluetooth, different wake-up sources, many peripherals, and much more. Additionally, the price difference is not a big deal, in my opinion. Once you move to the ESP32, you won’t want to go back to the ESP8266.

Wrapping Up

We hope you’ve found our analysis ESP32 vs ESP8266 useful.

Just to wrap up the main differences between the ESP32 and ESP8266:

  • The ESP32 is faster than the ESP8266;
  • The ESP32 comes with more GPIOs with multiple functions;
  • The ESP32 supports analog measurements on 18 channels (analog-enabled pins) versus just one 10-bit ADC pin on the ESP8266;
  • The ESP32 supports Bluetooth while the ESP8266 doesn’t;
  • The ESP32 is dual-core, and the ESP8266 is single core;
  • The ESP8266 is cheaper than the ESP32;
  • The ESP8266 has a wider community (although we don’t think that at this point, the difference is that relevant);
  • For many IoT and Wi-Fi projects, the ESP8266 can do the job for a lower price;
  • Both boards can be programmed with the Arduino core using Arduino IDE or other supported IDEs.
  • Both boards support MicroPython firmware.

You might like reading the following ESP8266 and ESP32 related articles to have an idea of the selection of the most popular ESP32 and ESP8266 development boards:

So, if you’re a beginner, should you get started with the ESP32 or the ESP8266? At this point, we definitely recommend getting started with the ESP32 instead of the ESP8266. However, if you already have an ESP8266 board, you can get started with that board and then make the shift to the ESP32.

We have a vast selection of projects with these boards in the Random Nerd Tutorials blog to help you get started:

Thanks for reading.

Analog JoyStick with Arduino

The Analog Joystick is similar to two potentiometers connected together, one for the vertical movement (Y-axis) and other for the horizontal movement (X-axis). The joystick also comes with a Select switch. It can be very handy for retro gaming, robot control or RC cars. So let’s understand how it works!dsc09428.jpg

Basics

The Arduino Uno or any other Arduino board that uses Atmega328 as the Microcontroller has ADC resolution of 10 bits. Hence the values on each analog channel can vary from 0 to 1023. Now connecting the VRx to A0 and VRy to A1 analog inputs respectively should show values as shown in the image below.

Joy diagram.jpeg

The home position for the stick is at ( x,y:511,511). If the stick is moved on X axis from one end to the other, the X values will change from 0 to 1023 and similar thing happens when moved along the Y axis. On the same lines you can read position of the stick anywhere in upper half hemisphere from combination of these values.

Hookup

0 Joystick with Arduino bb.png

So let’s print the values on the terminal so that we can verify the working!

Raw Sketch

  1. #define joyX A0
  2. #define joyY A1
  3.  
  4. void setup() {
  5. Serial.begin(9600);
  6. }
  7.  
  8. void loop() {
  9. // put your main code here, to run repeatedly:
  10. xValue = analogRead(joyX);
  11. yValue = analogRead(joyY);
  12.  
  13. //print the values with to plot or view
  14. Serial.print(xValue);
  15. Serial.print(\t);
  16. Serial.println(yValue);
  17. }

Mapping

It is usually not enough to read the analog values, you might want to map it to a display or any other interface. So let’s map these these values to a 8×8 led matrix. So that we can move the pixel with the joystick. You can easily change this to map to a graphic or a OLED display.Matrx LED and Joystick with Arduino bb.png

  1. #include “LedControl.h”
  2. #define joyX A0
  3. #define joyY A1
  4.  
  5. int xMap, yMap, xValue, yValue;
  6. LedControl lc=LedControl(12,11,10,1);
  7.  
  8. void setup() {
  9. Serial.begin(115200);
  10.  
  11. lc.shutdown(0,false);
  12. /* Set the brightness to a medium values */
  13. lc.setIntensity(0,8);
  14. /* and clear the display */
  15. lc.clearDisplay(0);
  16. }
  17.  
  18. void loop() {
  19. // put your main code here, to run repeatedly:
  20. xValue = analogRead(joyX);
  21. yValue = analogRead(joyY);
  22. xMap = map(xValue, 0,1023, 0, 7);
  23. yMap = map(yValue,0,1023,7,0);
  24. lc.setLed(0,xMap,yMap,true);
  25. lc.clearDisplay(0);
  26.  
  27. }

So as you see in the code above, the map() function can be used to map the ranges as you wish. Also notice that the Y axis map is inverted! So much to learn with a simple interface! Do checkout the retro ping-pong game built with the same setup.

 

 

 

int VRx = A0;
int VRy = A1;
int SW = 2;

int xPosition = 0;
int yPosition = 0;
int SW_state = 0;
int mapX = 0;
int mapY = 0;

void setup() {
  Serial.begin(9600); 
  
  pinMode(VRx, INPUT);
  pinMode(VRy, INPUT);
  pinMode(SW, INPUT_PULLUP); 
  
}

void loop() {
  xPosition = analogRead(VRx);
  yPosition = analogRead(VRy);
  SW_state = digitalRead(SW);
  mapX = map(xPosition, 0, 1023, -512, 512);
  mapY = map(yPosition, 0, 1023, -512, 512);
  
  Serial.print("X: ");
  Serial.print(mapX);
  Serial.print(" | Y: ");
  Serial.print(mapY);
  Serial.print(" | Button: ");
  Serial.println(SW_state);

  delay(100);
  
}

Mac 安裝Secp256k1 no secp256k1 in java.library.path

報錯

#報錯提示
no secp256k1 in java.library.path

解決方案

  1. 使用 brew 安裝構建依賴項
    brew install autoconf automake libtool berkeley-db4 pkg-config openssl boost boost-build libevent
  2. 下載secp256k1
    #隨便找個地方clone就行
    git clone git@github.com:bitcoin-core/secp256k1.git
  3. 構建secp256k1
    #以下步驟都在clone的secp256k1 中進行
    1. sh autogen.sh 
    #這一步與github 中的不同,github 的還是會出錯,視情況輸入命令吧
    2. ./configure --enable-module-recovery --enable-jni --enable-module-ecdh --enable-experimental
    3. make 
    4. make check 
    5. sudo make install
    6.

參考資料

DBeaver Ultimate 22.1 旗舰版激活方法

本站惯例:本文假定你知道DBeaver。不知道可以问问搜索引擎。

DBeaver是一款优秀的数据库管理工具,支持管理众多数据库产品,巴拉巴拉。

DBeaver Ultimate(简称DBeaverUE)支持MongoDBRedisApache Hive等,对比于DBeaver Enterprise多了连接云服务器的功能,但是需要付费使用。

这次要送的这份礼就是: DBeaverUE 22.1.0及以下版本(理论上适用于目前所有新老版本)的破解,可使用它来激活你手头上的DBeaverUE。

下载地址:
百度云下载(download link),提取码:hvx1
OneDrive(download link)

具体使用方法已写在压缩包的README.txt内,有什么问题可以给我提Issue或进QQ群:30347511讨论。

按照README.txt配置好之后,使用DBeaverUE专用激活码:

1
2
3
4
5
aYhAFjjtp3uQZmeLzF3S4H6eTbOgmru0jxYErPCvgmkhkn0D8N2yY6ULK8oT3fnpoEu7GPny7csN
sXL1g+D+8xR++/L8ePsVLUj4du5AMZORr2xGaGKG2rXa3NEoIiEAHSp4a6cQgMMbIspeOy7dYWX6
99Fhtpnu1YBoTmoJPaHBuwHDiOQQk5nXCPflrhA7lldA8TZ3dSUsj4Sr8CqBQeS+2E32xwSniymK
7fKcVX75qnuxhn7vUY7YL2UY7EKeN/AZ+1NIB6umKUODyOAFIc8q6zZT8b9aXqXVzwLJZxHbEgcO
8lsQfyvqUgqD6clzvFry9+JwuQsXN0wW26KDQA==

DBeaverUE有几点需要注意的:

  • windows 系统请使用ZIP包,下载链接:x64
  • mac 系统请使用DMG包,下载链接:intel / m1
  • linux 系统请使用.TAR.GZ包,下载链接:x64
  • DBeaver运行需要java,请自行安装!
  • 不要使用DBeaver自带的jre,它被人为阉割了。

22.1版本请在dbeaver.ini文件末尾添加一行:-Dlm.debug.mode=true

请自行安装jdk11,替换dbeaver.ini内由-vm指定的java路径,把地址换成自己安装的!

如果你的dbeaver.ini内没有-vm参数,请在首行添加你安装jdk的java路径:

1
2
-vm 
/path/to/your/bin/java

下面是国际惯例:

本项目只做个人学习研究之用,不得用于商业用途!

若资金允许,请购买正版,谢谢合作!

Spring Boot Admin的介绍及使用

Spring Boot 有一个非常好用的监控和管理的源软件,这个软件就是 Spring Boot Admin。该软件能够将 Actuator 中的信息进行界面化的展示,也可以监控所有 Spring Boot 应用的健康状况,提供实时警报功能。

主要的功能点有:

  • 显示应用程序的监控状态
  • 应用程序上下线监控
  • 查看 JVM,线程信息
  • 可视化的查看日志以及下载日志文件
  • 动态切换日志级别
  • Http 请求信息跟踪
  • 其他功能点……

可点击 https://github.com/codecentric/spring-boot-admin 更多了解 Spring-boot-admin。

创建Spring Boot Admin项目

创建一个 Spring Boot 项目,用于展示各个服务中的监控信息,加上 Spring Boot Admin 的依赖,具体代码如下所示。

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>de.codecentric</groupId>
<artifactId>spring-boot-admin-starter-server</artifactId>
<version>2.0.2</version>
</dependency>

创建一个启动类,具体代码如下所示。

  1. @EnableAdminServer
  2. @SpringBootApplication
  3. public class App {
  4. public static void main(String[] args) {
  5. SpringApplication.run(App.class, args);
  6. }
  7. }

在属性文件中增加端口配置信息:

server.port=9091

启动程序,访问 Web 地址 http://localhost:9091 就可以看到主页面了,这个时候是没有数据的,如图 1 所示。

Spring Boot Admin主页
图 1  Spring Boot Admin主页

将服务注册到 Spring Boot Admin

创建一个 Spring Boot 项目,名称为 spring-boot-admin-client。添加 Spring Boot Admin Client 的 Maven 依赖,代码如下所示。

<dependency>
<groupId>de.codecentric</groupId>
<artifactId>spring-boot-admin-starter-client</artifactId>
<version>2.0.2</version>
</dependency>

然后在属性文件中添加下面的配置:

server.port=9092
spring.boot.admin.client.url=http://localhost:9091

spring.boot.admin.client.url:Spring Boot Admin 服务端地址。

将服务注册到 Admin 之后我们就可以在 Admin 的 Web 页面中看到我们注册的服务信息了,如图 2 所示。

Spring Boot Admin主页(有数据)
图 2  Spring Boot Admin主页(有数据)

点击实例信息跳转到详细页面,可以查看更多的信息,如图 3 所示。

Spring Boot Admin详情
图 3  Spring Boot Admin详情

可以看到详情页面并没有展示丰富的监控数据,这是因为没有将 spring-boot-admin-client 的端点数据暴露出来。

在 spring-boot-admin-client 中加入 actuator 的 Maven 依赖,代码如下所示。

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

然后在属性文件中追加下面的配置:

management.endpoints.web.exposure.include=*

management.endpoints.web.exposure.include:暴露所有的 actuator 端点信息重启 spring-boot-admin-client,我们就可以在详情页面看到更多的数据,如图 4 所示。

Spring Boot Admin详情(有数据)
图 4  Spring Boot Admin详情(有数据)

监控内容介绍

自定义的 Info 信息、健康状态、元数据,如图 5 所示。

Spring Boot Admin数据展示(一)
图 5  Spring Boot Admin数据展示(一)

CPU、线程等信息如图 6 所示。

Spring Boot Admin数据展示(二)
图 6  Spring Boot Admin数据展示(二)

内存使用情况如图 7 所示。

Spring Boot Admin数据展示(三)
图 7  Spring Boot Admin数据展示(三)

配置信息如图 8 所示。

Spring Boot Admin数据展示(四)
图 8  Spring Boot Admin数据展示(四)

日志级别调整如图 9 所示。

Spring Boot Admin数据展示(五)
图 9  Spring Boot Admin数据展示(五)

Http请求信息如图 10 所示。

Spring Boot Admin数据展示(六)
图 10  Spring Boot Admin数据展示(六)

如何在Admin中查看各个服务的日志

Spring Boot Admin 提供了基于 Web 页面的方式实时查看服务输出的本地日志,前提是服务中配置了 logging.file。

我们在 spring-boot-admin-client 的属性文件中增加下面的内容:

logging.file=/Users/zhangsan/Downloads/spring-boot-admin-client.log

重启服务,就可以在 Admin Server 的 Web 页面中看到新加了一个 Logfile 菜单,如图 11 所示。

Spring Boot Admin日志
图 11  Spring Boot Admin 日志