In part 1 I presented the general idea for the Smart Greenhouse. In this part I am going to get my ESP32 development board up and running and start logging some data to an online database. I am going to describe what I did to:
- Set up a development environment for the ESP32
- Set up a MySQL database and write a simple REST API in order to store and retrieve data
- Write firmware to read data from the BME280 sensor and post it to the database
- Retrieve and visualize the data with Python
ESP32 development environment
As I mentioned in the previous post, I have had an ESP32 development kit (more specifically an ESP32-DevKitC) lying around for quite some time and this is the perfect opportunity for me to try it out.
The ESP32 is made by the Chinese company Espressif. They provide an extensive open-source SDK called the Espressif IoT Development Framework (ESP-IDF) that provides easy interfaces to all the features of the chip. I found a Get Started guide on their website with setup instructions for Windows, Linux and Mac. To start developing, you basically need a toolchain, a build system and (optionally, but recommended) the ESP-IDF. You can set all this up with whatever IDE you prefer, but they recommend using the official plugin/extension for either Eclipse or Visual Studio Code (VS Code). If you just want to get up and running quickly, there is also the option to download the Eclipse-based Espressif-IDE, which comes bundled with everything you need to build, flash and monitor code on the target. You can download it here.
Ever since I started doing embedded development I have almost exclusively used Eclipse-based vendor-specific IDEs, but I have been wanting to try Visual Studio Code (and IntelliSense) instead. So I downloaded and installed VS Code and followed Espressif’s YouTube video for setting up the ESP-IDF extension.
Database and REST API
Now, before writing any firmware, I wanted to set up a temporary database and API on my webhost where I can send HTTP POST and GET requests to store and retrieve data.
I logged on to my webhost, created a new MySQL database, and then opened up phpMyAdmin to create a new climate_data table with the following fields:
- id (PRIMARY_KEY)
- timestamp
- temperature
- humidity
- pressure
Next, I created a simple PHP script that can handle POST requests to store temperature, humidity and pressure data in the database, and GET requests to retrieve the data and present it in JSON format. Now I have a simple endpoint that the ESP32 can communicate with.
Firmware
I returned to VS Code and created a new ESP-IDF project from the “Hello world” example and started looking for a main() function. However, the only source file in the project (main/hello_world_main.c) only contained the function app_main(). I searched the documentation, and it turns out that before app_main() is invoked, ESP-IDF handles a bunch of initialization, creates the main task and starts the FreeRTOS scheduler. The main task calls the app_main() function, which is then considered the entry point of the application.
I decided to create two tasks: A climate_task for reading data from the BME280 and pushing it to a queue, and a networking_task that pops data from the queue and transmits it to the REST API with an HTTP POST request. I also created a custom data type, climate_data_t, in order to decouple the networking task from the climate task and the BME280. In my initial tests of the BME280 I was not too impressed with its measurement accuracy, so in case I decide to switch sensor later on, I will only have to make changes to the climate task – not the networking task. A task diagram is pictured below:
I renamed hello_world_main.c to main.c, created climate_data.h for the data type and created a new folder main/tasks/ where I placed a header and source for each of the tasks:
greenhouse
|--main/
| |--tasks/
| | |--climate_task.c
| | |--climate_task.h
| | |--networking_task.c
| | |--networking_task.h
| |--climate_data.h
| |--CMakeLists.txt
| |--main.c
|--CMakeLists.txt
|--README.mdI edited main/CMakeLists.txt to make it build the renamed main.c file and the two task source files:
set(TASK_SOURCES
tasks/wifi_task.c
tasks/climate_task.c)
idf_component_register(SRCS main.c ${TASK_SOURCES}
INCLUDE_DIRS ".")
target_compile_options(${COMPONENT_LIB} PRIVATE "-Wno-format")
Then in app_main() I created a queue of climate_data_t along with the climate and networking tasks which both take the queue as an argument:
#include "climate_data.h"
//...
void app_main(void)
{
QueueHandle_t climate_queue = xQueueCreate(16, sizeof(climate_data_t));
xTaskCreate(networking_task,
"Network task",
configMINIMAL_STACK_SIZE*4*10,
climate_queue,
LOW_PRIORITY,
NULL);
xTaskCreate(climate_task,
"Climate task",
configMINIMAL_STACK_SIZE*4*10,
climate_queue,
LOW_PRIORITY,
NULL);
}Next up is implementing the tasks.
Climate task
To interface with the BME280, I cloned the official driver from Bosch’s Github repository. In the ESP-IDF documentation, pieces of modular, stand-alone code are referred to as “components” and are added to the project by registering the component to the build system. Basically, to add a component you specify in the root CMakeLists.txt where the component is located (e.g. by setting the EXTRA_COMPONENT_DIRS variable) and then you create a CMakeLists.txt inside the component directory, where you register the component. The two CMakeLists.txt now look like this:
cmake_minimum_required(VERSION 3.16)
set(EXTRA_COMPONENT_DIRS custom_components)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(greenhouse)idf_component_register(SRCS "bme280.c"
INCLUDE_DIRS ".")The BME280 functions as an I2C slave, so I set the ESP32 up as a master using the ESP-IDF I2C driver. Also, the BME280 driver requires pointers to three hardware-dependent functions in its initialization struct: i2c_read(), i2c_write() and delay_us(). I implemented these three functions, connected the sensor’s SDA and SCL pins to pin GPIO21 and GPIO22 on the ESP32 and the sensor was good to go.
The climate task itself is fairly simple. Before starting the task’s super loop we have to initialize the I2C peripheral and the BME280. Then in the super loop we read some data from the sensor, convert them to a climate_data_t and push it to the queue. Then we just suspend the task until we want another reading. Note that, for brevity, I have omitted logging, error checking, function definitions, etc. in the listing below as well as later in the post.
static struct bme280_dev bme280 = {...};
void climate_task(void* params)
{
QueueHandle_t climate_queue = (QueueHandle_t) params;
init_i2c_master();
bme280_init(&bme280);
while (1)
{
struct bme280_data data = read_all_bme_data(&bme280);
climate_data_t standardized_data = convert_bme_data(&data);
xQueueSend(climate_queue, &standardized_data, portMAX_DELAY);
vTaskDelay(minutes_to_ticks(15));
}
}Networking task
For the networking task I took inspiration from a few ESP-IDF example projects: fast_scan to connect to my home WiFi and the http_rest_with_url function from esp_http_client_example.
To connect to a WiFi access point, you basically go through the following steps:
- Initialize the network interface (esp_netif) and WiFi driver (esp_wifi) with default configurations.
- Start the default event loop task with the esp_event component
- Register a WiFi event handler and an IP event handler to the event loop and implement the callback function.
- Configure the device as a WiFi station in both the network interface and WiFi driver
- Configure the WiFi scan settings (SSID, password, etc.) and start the WiFi driver.
Now, you will receive callbacks for any events you have registered in the event loop, e.g when the WiFi driver is started (so you can proceed to connect to the AP), when the WiFi disconnects and when you get an IP address from the DHCP server. I recommend looking through the fast_scan example to get a better idea of how it works in practice.
The HTTP client is quite simple to set up, as you just have to initialize it with a client configuration struct (which includes a function pointer to your event handler), but it does have more events for you to handle. To actually make a request, you just set the host URL, method, header, body, authentication credentials, etc. and then call esp_http_client_perform() and wait for a response which triggers an event.
After all the initialization, the task (given below in simplified form) just waits for data from the climate task and then sends the data in a POST request to the REST API:
typedef enum
{
AWAIT_DATA,
TRANSMIT
} wifi_state_t;
static wifi_state_t state;
bool wifi_connected;
void wifi_task(void* params)
{
QueueHandle_t climate_queue = (QueueHandle_t) params;
climate_data_t climate_data;
esp_http_client_handle_t client = esp_http_client_init(&config);
netif_wifi_init();
while (1)
{
switch (state)
{
case AWAIT_DATA:
xQueueReceive(climate_queue, &climate_data, portMAX_DELAY);
state = TRANSMIT;
break;
case TRANSMIT:
if (!wifi_connected)
{
// Wait for WiFi to connect
vTaskDelay(pdMS_TO_TICKS(5*1000));
break;
}
post_climate_data(client, climate_data);
state = AWAIT_DATA;
break;
}
}
}The wifi_connected bool is set and cleared in the WiFi event handler and the device automatically tries to reconnect if it loses WiFi connection.
After finishing up the firmware and flashing it to the ESP32, I placed the device in a shaded area in the greenhouse and powered it with a USB power supply connected to the mains with an extension cord. I plan to power it from a battery/solar panel setup later on, but this will do for testing.
Visualizing the data with Python
I logged onto phpMyAdmin and could see data being pushed to the database every 15 minutes. To get a nice graph of the temperature fluctuations over the course of 24 hours, I wrote a small Python script to GET the data using the REST API and plot it with matplotlib:
import requests
import matplotlib.pyplot as plt
if __name__ == '__main__':
# Retrieve JSON-encoded data via REST API
URL = "https://url.to/my/api"
params = {"hours": 24}
r = requests.get(URL, params)
json_data = r.json()
timestamps = [ item['timestamp'] for item in json_data ]
temps = [ float(item['temperature']) for item in json_data ]
timestamps.reverse()
temps.reverse()
# Plot data
fig = plt.figure()
fig.suptitle("Temperature over 24 hours")
plt.plot(timestamps, temps)
plt.xticks(range(0, len(temps), len(temps)//5), rotation=45)
plt.tight_layout()
plt.ylabel("degrees C")
plt.show()
After letting the device log data for 24 hours, I ran the script – et voilá! A nice graph showing a minimum temperature of 17 °C at 7:30 in the morning and a maximum temperature of 29 °C at 18:00 in the evening.
In part 3 I am going to set up a more permanent power supply with a solar panel and a battery, measure the power consumption of the ESP32 and figure out how to optimize it.
1 thought on “Project Smart Greenhouse (Part 2): Data collection with ESP32”