Wiznet makers

Step 4.0: Using the Open Food Facts JSON API to get product information by barcode

Open Food Facts is created by a non-profit association and provides a worldwide food products database. Also, the Open Food Facts JSON API lets the user search a product by barcode and obtain provided product characteristics by making an HTTP GET request.

Although Open Food Facts only covers food products, it has a vast database, including products worldwide. Therefore, I decided to utilize the Open Food Facts JSON API to obtain product information by barcode to demonstrate the features of my shopping assistant.

If you want to obtain product information by barcodes of products other than foods, you can utilize some overall product database services with paid subscription plans, such as Barcode Lookup.

📁 barcode.php

⭐ Include the class.php file.

include_once "class.php";

⭐ Define the _product object of the product class extending the user class.

$_product = new product();
$_product->__init__($conn);

⭐ If all required HTML (GET) parameters are received:

⭐ Check whether the given table is in the MariaDB (MySQL) database.

⭐ If so, make an HTTP GET request to the Open Food Facts JSON API to obtain the product information with the given barcode.

⭐ According to the selected command by the user, add or remove the given product information to/from the user's database table.

if(isset($_GET["table"]) && isset($_GET["barcode"]) && isset($_GET["com"])){	
	// Check whether the given table is in the MySQL database.
	if($_product->check_table($_GET["table"])){
		// Make an HTTP GET request to the Open Food Facts JSON API to obtain the product information with the given barcode.
		$product_info = $_product->get_product_info($_GET["barcode"]);
		// According to the selected command by the user, add or remove the given product to/from the user's database table.
		if($_GET["com"] == "add"){
			$_product->insert_product($_GET["table"], $_GET["barcode"], $product_info["name"], $product_info["ingredients"], $product_info["price"]);
			echo "Given Product Added to the Cart Successfully!";
		}else if($_GET["com"] == "remove"){
			$_product->delete_product($_GET["table"], $_GET["barcode"]);
			echo "Given Product Removed from the Cart Successfully!";
		}
	}else{
		echo "No Table Found!";
	}
}

 

Step 4.1: Designing the home page and the sign-in page

📁 new_account.php

⭐ Include the class.php file.

include_once "class.php";

⭐ Define the _user object of the user class.

$_user = new user();
$_user->__init__($conn);

⭐ Create a new user account if the customer fills out the sign-up form accurately.

if(isset($_GET['firstname']) && isset($_GET['lastname']) && isset($_GET['email']) && isset($_GET['username']) && isset($_GET['password']) && isset($_GET['c_password'])){
	$_user->add_new_account($_GET['firstname'], $_GET['lastname'], $_GET['email'], $_GET['username'], $_GET['password'], $_GET['c_password']);
}

⭐ Sign into an existing account if the user fills out the sign-in form correctly.

if(isset($_GET['u_username']) && isset($_GET['u_password'])){
	$_user->user_login_request($_GET['u_username'], $_GET['u_password']);
}

📁 Index.php

⭐ If the user has already signed in, redirect the customer to the user interface (dashboard).

if(isset($_SESSION["username"]) && isset($_SESSION["email"])){
	header('Location: ./dashboard.php');
	exit();
}

⭐ Design the sign-up form and the home page of the e-commerce web application.

You can inspect and download the index.php file below.

📁 login.php

⭐ If the user has already signed in, redirect the customer to the user interface (dashboard).

if(isset($_SESSION["username"]) && isset($_SESSION["email"])){
	header('Location: ./dashboard.php');
	exit();
}

⭐ Design the sign-in page for returning customers.

📁 Index.css

You can inspect and download the index.css file below.

 

 

 

 

Step 4.2: Designing the user interface (dashboard) showing the account QR code

📁 dashboard.php

⭐ If the user signed into an existing account successfully, proceed to load the user dashboard.

if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
	header('Location: ./');
	exit();
}

⭐ If the user requests to log out, remove and destroy all session variables.

⭐ Then, redirect the user to the home page.

if(isset($_GET["logout"])){
	// Remove and destroy all session variables.
	session_unset();
	session_destroy();
	// Go to the home page.
	header('Location: ./');
	exit();
}

⭐ Design the user dashboard (interface) consists of:

• a self-updating product list in the current cart,
• total price indicator and the Checkout button,
• the unique user (account) token,
• the Logout button,
• the Account Information section,
• the Previous Orders button.

⭐ Design the hidden checkout form to pass the estimated total cart amount to the checkout page.

<form id="hidden_checkout" action="checkout.php" method="post" target="_blank" style="display:none;">
<input id="checkout_price" name="checkout_price" type="hidden" value="$0">
</form>

📁 dashboard_updates.php

⭐ Include the class.php file.

⭐ If the user did not sign into an existing account, redirect the user to the home page.

include_once "class.php";

if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
	header('Location: ./');
	exit();
}

⭐ Define the _product object of the product class extending the user class.

$_product = new product();
$_product->__init__($conn);

⭐ If the update URL (GET) parameter is received:

⭐ Retrieve the product information of the current cart from the user's database table as a list.

⭐ Generate HTML table rows consisting of the retrieved product information and the Ask ChatGPT button redirecting to the ChatGPT recommendations page of the product.

⭐ Then, create a JSON object from the generated HTML table rows and the evaluated total cart price.

⭐ Print the created JSON object.

if(isset($_GET["update"])){
	$p_barcode = []; $p_name = []; $p_ingredients = []; $p_price = []; $p_number = []; $total_price = [];
	list($p_barcode, $p_name, $p_ingredients, $p_price, $p_number, $total_price) = $_product->get_current_products($_SESSION["user_token"]);
	$list = "<tr><th>Product</th><th>Barcode</th><th>Ingredients</th><th>Price($)</th><th>Unit</th><th>Counsel</th></tr>";
	for($i=0; $i<count($p_barcode); $i++){
		$list .= '<tr>
					<td>'.$p_name[$i].'</td>
					<td>'.$p_barcode[$i].'</td>
					<td>'.$p_ingredients[$i].'</td>
					<td>'.$p_price[$i].'</td>
					<td>'.$p_number[$i].'</td>
					<td><a href="ChatGPT/?product='.$p_name[$i].'" target="_blank"><button><i class="fa-solid fa-comment-dots"></i> Ask ChatGPT</button></a></td>
				  </tr>
				 ';   
	}
	
	// Create a JSON object from the generated HTML table rows consisting of the product information and the evaluated total cart price.
	$data = array("list" => $list, "total_price" => "$".$total_price["total_price"]);
	$j_data = json_encode($data);
    
	// Return the recently generated JSON object.
	echo($j_data);
}

📁 Index.js (jQuery and AJAX)

⭐ Every 5 seconds, make an HTTP GET request to the dashboard_updates.php file.

⭐ Then, decode the received JSON object to obtain the HTML table rows consisting of the product information of the current cart from the user's database table and the estimated total cart price.

⭐ Assign the obtained table rows and the total cart price to the corresponding HTML elements on the user dashboard so as to showcase the recent products added to the cart and inform the customer of the calculated total cart price (amount).

setInterval(function(){
	$.ajax({
		url: "./assets/dashboard_updates.php?update",
		type: "GET",
		success: (response) => {
			// Decode the obtained JSON object.
			const data = JSON.parse(response);
			// Assign the HTML table rows as the current product list in the cart. 
			$(".products table").html(data.list);
			// Assign the evaluated total cart price (amount).
			$("#total_price").html(data.total_price);
		}
	});
}, 5000);

⭐ When the user clicks the Checkout button, redirect the customer to the checkout page and transfer the evaluated total cart price via the hidden HTML form.

$(".info").on("click", "#checkout", () => {
	var total_price = $("#total_price").text();
	$("#checkout_price").val(total_price);
	$("#hidden_checkout").submit();
});

 

 

 

Step 4.3: Displaying the previous orders when requested

📁 previous_orders.php

⭐ Include the class.php file.

include_once "assets/class.php";

⭐ Define the _product object of the product class extending the user class.

$_product = new product();
$_product->__init__($conn);

⭐ If the user signed into an existing account successfully, proceed to load the previous orders page.

if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
	header('Location: ./');
	exit();
}

⭐ Design the previous orders page to display the purchased product list of each previous order as an HTML list.

$_product->get_previous_orders($_SESSION["user_token"]);

 

Step 4.4: Designing the checkout (payment) page

After creating the checkout page and HTML form to get the payment information (credit/debit card), I did not integrate a payment processing service since I do not have partner access as a business owner to general payment services, such as Visa.

Nonetheless, it is easy to send the received payment information to a payment processing service if you have a local business with partner access, as explained at Visa Checkout.

📁 checkout.php

⭐ Include the class.php file.

include_once "assets/class.php";

⭐ Define the _product object of the product class extending the user class.

$_product = new product();
$_product->__init__($conn);

⭐ If the user signed into an existing account successfully, proceed to load the checkout page.

if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
	header('Location: ./');
	exit();
}

⭐ If the user places an order by entering the requested credit/debit card information, execute the user_checkout function, explained in the previous steps.

if(isset($_GET["c_number"]) && isset($_GET["c_name"]) && isset($_GET["c_date"]) && isset($_GET["c_verify"])){
	$_product->user_checkout($_SESSION["user_token"], $_SESSION["email"], $_SESSION["name"]);
}

⭐ Design the checkout page and the checkout HTML form.

⭐ Check whether the total cart price variable is received via the hidden HTML form on the dashboard page.

if(isset($_POST["checkout_price"])){
	echo $_POST["checkout_price"];
}else{
	echo "$0";
}

 

Step 4.5: Integrating ChatGPT into the web application via the OpenAI API

As explained in the previous steps, I employed the OpenAI API to generate ChatGPT-powered recommendations for the given product instead of coercing local businesses to construct large data sets of their products.

In that regard, I was able to provide insightful product suggestions by merely making API calls to the OpenAI API.

While making a cURL call (HTTP request) to the OpenAI API, the web application passes five questions for the gpt-3.5-turbo model:

• What is the nutritional value of <_product_>?
• What should I purchase with <_product_>?
• Can you teach me a recipe with <_product_>?
• How should I serve <_product_>?
• Is there a more affordable and healthy option than <_product_>?

Since ChatGPT accepts system commands in the text format to alter the generated response structure, I added a simple command to add the exact question with the question number before each answer. In that regard, I was able to modify the retrieved response from ChatGPT to add line breaks between answers.

{"role": "user", "content": "Please add the exact question at the beginning of the answer with the question number."}

📁 Index.php

⭐ Include the class.php file.

include_once "../assets/class.php";

⭐ Define the _product object of the product class extending the user class.

$_product = new product();
$_product->__init__($conn);

⭐ If the user signed into an existing account successfully, proceed to load the ChatGPT recommendations page.

if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
	header('Location: ./');
	exit();
}

⭐ If the customer requests to get ChatGPT-powered recommendations for an eligible product, execute the chatgpt_get_suggestion function so as to make a cURL call (HTTP request) to the OpenAI API in order to get suggestions regarding the given product from ChatGPT.

$suggestions = "<h2>Please enter a product name.</h2>";
$questions = ["Please enter a product name."];
if(isset($_GET["product"]) && $_GET["product"] != "Not Found!"){
	list($suggestions, $questions) = $_product->chatgpt_get_suggestion($_GET["product"]);
}

⭐ Design the ChatGPT recommendations page by utilizing the ChatGPT theme.

⭐ Print the defined questions and the modified ChatGPT-powered answers to inform the customer.

for($i=0;$i<count($questions);$i++){
	echo '<h2><i class="fa-regular fa-comment-dots"></i> '.$questions[$i].'</h2>';
}

...

echo $suggestions;

📁 Index.css

You can download and inspect the index.css file of the ChatGPT folder below.

 

 

Step 4.6: Setting and running the e-commerce web application on LattePanda 3 Delta

Since I wanted to build a budget-friendly AIoT shopping assistant not dependent on cloud or hosting services, I decided to host my web application on LattePanda 3 Delta 864. Therefore, I needed to set up a LAMP web server.

LattePanda 3 Delta is a pocket-sized hackable computer that provides ultra performance with the Intel 11th-generation Celeron N5105 processor.

Plausibly, LattePanda 3 Delta can run the XAMPP application. So, it is effortless to create a server with a MariaDB database on LattePanda 3 Delta.

Also, I utilized a DFRobot 8.9" IPS touch display to demonstrate my e-commerce web application features.

 

#️⃣ First of all, install and set up the XAMPP application.

#️⃣ Then, go to the XAMPP Control Panel and click the MySQL Admin button.

#️⃣ Once the phpMyAdmin tool pops up, create a new database named shopping_assistant_users.

 

 

#️⃣ After adding the database successfully, go to the SQL section to create a MySQL database table named users with the required data fields.

CREATE TABLE `users`(		
	id int AUTO_INCREMENT PRIMARY KEY NOT NULL,
	firstname varchar(255) NOT NULL,
	lastname varchar(255) NOT NULL,
	username varchar(255) NOT NULL,
    `password` varchar(255) NOT NULL,
	email varchar(255) NOT NULL,
    token varchar(255) NOT NULL,
	qr_code varchar(255) NOT NULL,
    successful_order int NOT NULL
);

 

 

 

Step 5: Configuring NUCLEO-F439ZI with the modified W5300 TOE SHIELD firmware

Since the STM32 Nucleo-144 boards need to be flashed with the compatible Wiznet5K firmware to control W5300 TOE SHIELD and program it in MicroPython, I had to upload the official Wiznet5K firmware to the NUCLEO-F439ZI development board.

Plausibly, WIZnet provides official pre-built firmwares for these Nucleo-144 boards:

• NUCLEO_F429ZI
• NUCLEO_F439ZI
• NUCLEO_F722ZE
• NUCLEO_F756ZG
• NUCLEO_F767ZI

Although there is a pre-built firmware for NUCLEO-F439ZI, I decided to compile my Wiznet5K MicroPython firmware in order to change some minor pin settings due to overlapping.

You can download my updated firmware below — firmware_updated.hex.

Since I have a secondary operating system (Ubuntu) on my LattePanda 3 Delta, I was able to use it to build my Wiznet5K firmware.

#️⃣ First, open the terminal to install the required Python modules.

sudo apt-get install git
    sudo apt-get install make
    sudo apt-get install gcc
	sudo apt-get install gcc-arm-none-eabi

#️⃣ Then, clone the official Wiznet5K MicroPython project from GitHub by entering the following Git command.

git clone https://github.com/Wiznet/W5300-TOE-MicroPython.git

 

#️⃣ After installing the MicroPython project, I modified some pin settings for NUCLEO-F439ZI in the mpconfigboard.h file.

/libraries/ports/stm32/boards/NUCLEO_F439ZI/mpconfigboard.h

 

 

#️⃣ Finally, I built my modified firmware for the NUCLEO-F439ZI development board attached to W5300 TOE SHIELD. Do not forget to change the board name and the chip model for different combinations.

    cd W5300-TOE-MicroPython/libraries/mpy-cross
  
	make

	cd ../ports/stm32
	
	make BOARD=NUCLEO-F439ZI MICROPY_PY_WIZNET5K=5300

 

 

 

#️⃣ Navigate to the build-NUCLEO-F439ZI folder available in /libraries/ports/stm32, and the compiled firmware file should be there — firmware.hex.

 

After building my firmware version successfully, I employed the STM32CubeProgrammer on Windows to upload the modified firmware to NUCLEO-F439ZI.

#️⃣ First, click the Open File button to open the modified firmware file — firmware_updated.hex.

 

 

#️⃣ Then, connect NUCLEO-F439ZI to the computer via a USB cable and click the Connect button.

 

#️⃣ Finally, click the Download button to flash NUCLEO-F439ZI with the given firmware.

 

Step 6: Installing and modifying MicroPython libraries on NUCLEO-F439ZI

After flashing my Wiznet5K firmware to the NUCLEO-F439ZI development board, I utilized Thonny to program NUCLEO-F439ZI attached to W5300 TOE SHIELD.

However, since the Wiznet5K firmware does not include the required modules for some components, I needed to upload modules to NUCLEO-F439ZI and modify them if the default settings are not compatible with W5300 TOE SHIELD.

⚠️ Notice:

Even though the W5300 TOE SHIELD datasheet mentions one available hardware serial port (USART6), NUCLEO-F439ZI has two eligible hardware serial ports (USART2 and USART6), as shown in the STM32 Nucleo-144 user manual. Therefore, I was able to connect two components (the scanner module and the thermal printer), requiring serial communication.

#️⃣ To program NUCLEO-F439ZI on Thonny, go to Tools ➡ Select interpreter... and choose MicroPython (generic) with the given STLink Virtual COM Port.

#️⃣ Then, via Thonny's file explorer, select the MicroPython device's flash folder to upload or update files.

 

 

 

 

#️⃣ To make HTTP requests easily, upload the urequests.py file (module) to NUCLEO-F439ZI via Thonny's file explorer.

#️⃣ To get the pre-defined functions for the W5300 chip, upload the wiznet_conf.py file.

#️⃣ To control the SSD1309 transparent display, upload the ssd1309.py file.

#️⃣ To display fonts provided by the ssd1309 module on the transparent screen, upload the xglcd_font.py file.

 

 

 

 

#️⃣ To control the tiny (embedded) thermal printer, upload the Adafruit_Thermal.py file.

Since this module utilizes USART1 as the default hardware serial port and it is not available on NUCLEO-F439ZI, I needed to modify the Adafruit_Thermal.py file to utilize USART6 instead. I also added the required port settings for the thermal printer.

self.uart = UART(6, 9600, bits=8, parity=None, stop=1)

I also modified the data_end parameter for the printed bitmap images to align them perfectly.

data_end = header.file_size - 0

 

 

 

#️⃣ Finally, create a folder named assets on the device to upload the required image and font files.

You can download the assets folder with the added files below.

 

 

Step 7: Programming NUCLEO-F439ZI to scan data and communicate with the e-commerce application

After uploading all required MicroPython modules and files to NUCLEO-F439ZI, I created the main.py file to program W5300 TOE SHIELD. Since NUCLEO-F439ZI runs the main.py file automatically, I did not encounter any problems while initiating the shopping assistant (device).

 

⭐ Include the required modules.

⭐ Define the shopping_assistant class and its functions.

⭐ In the __init__ function:

⭐ Define the W5300 object and the static IP address settings depending on your network settings.

⭐ Define the required print and hardware serial port settings (USART6) for the tiny (embedded) thermal printer.

⭐ Define the required hardware serial port settings (USART2) for the GM77 barcode and QR code scanner.

⭐ Define the required SPI settings for the SSD1309 OLED transparent display.

⭐ Get the fonts stored in the assets folder.

⭐ Define the control button pins.

⭐ Define the RGB LED pins.

⭐ Initialize the SSD1309 OLED transparent display.

class shopping_assistant:
    def __init__(self):
        # Define the W5300 object and the static IP address settings.
        self.w5300 = wiznet5k_w5300()
        self.w5300.w5300_set_ip('0.0.0.0','0.0.0.0','0.0.0.0','0.0.0.0')
        # Define the required print and hardware serial port settings for the tiny (embedded) thermal printer.
        self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
        # Define the required hardware serial port settings for the GM77 barcode and QR code scanner.
        self.scanner = UART(2, 9600, bits=8, parity=None, stop=1)
        self.user_token = ""
        self.new_product_barcode = ""
        # Define the required settings for the SSD1309 OLED transparent display. (SCK: PA5, MOSI: PA7)
        spi = SPI(1, SPI.MASTER, baudrate=10000000)
        self.display = Display(spi, dc=Pin("A4"), cs=Pin("C6"), rst=Pin("C7"))
        # Define the given fonts.
        self.bold = XglcdFont('assets/Unispace12x24.c', 12, 24)
        self.light = XglcdFont('assets/Bally5x8.c', 5, 8)
        # Define the control button pins.
        self.button_A = Pin("C0", Pin.IN, Pin.PULL_UP)
        self.button_B = Pin("C3", Pin.IN, Pin.PULL_UP)
        self.button_C = Pin("C2", Pin.IN, Pin.PULL_UP)
        # Define the RGB LED pins.
        self.red = Pin("A1", Pin.OUT_PP)
        self.blue = Pin("A0", Pin.OUT_PP)
        self.green = Pin("F9", Pin.OUT_PP)
        self.adjust_color([0,0,0])
        sleep(2)
        # Initialize the SSD1309 OLED transparent display.
        self.product_menu_activate = False
        self.change_layout("home")

⭐ In the read_QR_barcode function:

⭐ If the scanner module reads a barcode or QR code successfully, get the data packet via serial communication.

⭐ Decode and modify the received data packet to obtain the scanned information.

⭐ If the scanned information includes the user command and the unique user token, activate the cart and register the given user token.

⭐ If the scanned information includes the finished command and the received user token corresponds to the registered user token, discard the registered user token to deactivate the cart.

⭐ Then, notify the customer via the thermal printer.

⭐ If the cart is activated and the scanned information includes a product barcode, save the given product barcode and initiate the product menu (interface) to allow the customer to select a command (add or remove) before transferring the given barcode to the e-commerce web application.

def read_QR_barcode(self):
        scanned_data = self.scanner.readline()
        # Decode and modify the received data packet to obtain the user (account) token with the given command or the new product barcode.
        if(type(scanned_data) is not type(None)):
            decoded_data = scanned_data.decode("utf_8")
            decoded_data = decoded_data.replace("\r", "")
            print("\nScanned: " + decoded_data)
            # Get the user token.
            if(decoded_data.find("user%") >= 0):
                self.user_token = decoded_data.split("%")[1]
                print("\nYour cart registered successfully!")
                print("Registered Token: " + self.user_token)
                self.change_layout("register")
                sleep(10)
                self.change_layout("scan")
            # After getting the finished command, discard the registered user token to deactivate the cart.
            elif(decoded_data.find("finished%") >= 0):
                given_token = decoded_data.split("%")[1]
                if(given_token == self.user_token):
                    print("\nPayment Received Successfully!")
                    print("Your cart discarded successfully!")
                    self.change_layout("payment")
                    sleep(2)
                    # Notify the customer via the thermal printer.
                    self.print_status(True)
                    self.change_layout("home")
                    self.user_token = ""
            # Get the product barcode.
            else:
                if(self.user_token != ""):
                    self.new_product_barcode = decoded_data
                    print("New Product Barcode: " + self.new_product_barcode)
                    self.product_menu_activate = True
                    if(self.product_menu_activate == True):
                        self.change_layout("barcode")
                        while(self.product_menu_activate == True):
                            self.product_menu()
                else:
                    print("\nPlease scan your unique account QR code to register your cart.") 
        sleep(1)

⭐ In the make_get_request function:

⭐ Make an HTTP GET request to the e-commerce web application by utilizing the registered user token, the given barcode, and the selected command by the customer.

http://192.168.1.22/AIoT_Shopping_Assistant/assets/barcode.php?table=kutlu123_!t3*b1450zKD&barcode=80135876&com=add

http://192.168.1.22/AIoT_Shopping_Assistant/assets/barcode.php?table=kutlu123_!t3*b1450zKD&barcode=80135876&com=remove

⭐ Then, print the response of the web application.

def make_get_request(self, barcode, com="add"):
        path = "http://192.168.1.22/AIoT_Shopping_Assistant/assets/barcode.php?table={}&barcode={}&com={}".format(self.user_token, barcode, com)
        # Make an HTTP GET request.
        request = urequests.get(path)
        print("\nURL => "+path)
        print("App Response => ")
        print(request.text)
        sleep(1)

⭐ In the product_menu function:

⭐ If the user presses the control button (A), make an HTTP GET request to the web application by adding the add command.

⭐ If the user presses the control button (B), make an HTTP GET request to the web application by adding the remove command.

⭐ After making the request, close the product menu (interface).

def product_menu(self):
        print("Press: Button (A) -> Add | Button (B) -> Remove")
        sleep(1)
        if(self.button_A.value() == False):
            self.make_get_request(self.new_product_barcode, "add")
            self.change_layout("add")
            sleep(10)
            self.change_layout("scan")
            self.product_menu_activate = False
        if(self.button_B.value() == False):
            self.make_get_request(self.new_product_barcode, "remove")
            self.change_layout("remove")
            sleep(10)
            self.change_layout("scan")
            self.product_menu_activate = False

⭐ In the print_status function:

⭐ If the user presses the control button (C), inform the user of the current device status by printing a receipt via the tiny (embedded) thermal printer.

⭐ Change the thermal printer hardware settings (heat time, heat dots, heat interval) to obtain smoother prints depending on the targeted print feature (image, text, inverted, etc.).

⭐ Also, if the customer places an order successfully and scans the payment confirmation QR code, notify the user by printing an invoice via the thermal printer.

def print_status(self, payment=False):
        if(self.button_C.value() == False):
            print("\nPrinting the device status...")
            # Change the thermal printer hardware settings to obtain smoother prints depending on the targeted feature.
            if(self.user_token == ""):
                self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
                self.printer.printBMPImage('assets/printer_chatgpt.bmp')
                self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
                self.printer.justify('C')
                self.printer.setSize('L')
                self.printer.println("AIoT")
                self.printer.println("Shopping")
                self.printer.println("Assistant")
                self.printer.println("Status\n\n")
                self.printer.justify('L')
                self.printer.setSize('S')
                self.printer.boldOn()
                self.printer.println("Please scan")
                self.printer.println("your unique")
                self.printer.println("account QR code")
                self.printer.println("to activate your")
                self.printer.println("cart and start")
                self.printer.println("shopping!\n\n")
                self.printer.boldOff()
                self.printer.justify('R')
                self.printer.setSize('M')
                self.printer.inverseOn()
                self.printer.println(" Have a ")
                self.printer.println(" Great ")
                self.printer.println(" Day :) \n")
                self.printer.inverseOff()
                self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
                self.printer.printBMPImage('assets/printer_scan.bmp')
                self.printer.feed(5)
            elif(self.user_token != ""):
                self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
                self.printer.printBMPImage('assets/printer_chatgpt.bmp')
                self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
                self.printer.justify('C')
                self.printer.setSize('L')
                self.printer.println("AIoT")
                self.printer.println("Shopping")
                self.printer.println("Assistant")
                self.printer.println("Status\n\n")
                self.printer.justify('L')
                self.printer.setSize('S')
                self.printer.boldOn()
                self.printer.println("Your cart is")
                self.printer.println("registered and")
                self.printer.println("initialized")
                self.printer.println("successfully.")
                self.printer.println("Please scan a")
                self.printer.println("product barcode")
                self.printer.println("to change")
                self.printer.println("the cart")
                self.printer.println("items!\n\n")
                self.printer.println("Registered Token:")
                self.printer.doubleHeightOn()
                self.printer.println(self.user_token)
                self.printer.println("\n")
                self.printer.doubleHeightOff()
                self.printer.boldOff()
                self.printer.justify('R')
                self.printer.setSize('M')
                self.printer.inverseOn()
                self.printer.println(" Have a ")
                self.printer.println(" Great ")
                self.printer.println(" Day :) \n")
                self.printer.inverseOff()
                self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
                self.printer.printBMPImage('assets/printer_cart.bmp')
                self.printer.feed(5)
        # If the customer places an order successfully:
        if(payment == True):
            self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
            self.printer.printBMPImage('assets/printer_chatgpt.bmp')
            self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
            self.printer.justify('C')
            self.printer.setSize('L')
            self.printer.println("AIoT")
            self.printer.println("Shopping")
            self.printer.println("Assistant")
            self.printer.println("Status\n\n")
            self.printer.justify('L')
            self.printer.setSize('S')
            self.printer.boldOn()
            self.printer.println("Payment for")
            self.printer.println("your latest")
            self.printer.println("order is received")
            self.printer.println("successfully via")
            self.printer.println("your dashboard")
            self.printer.println("on the web")
            self.printer.println("application.")
            self.printer.println("Please scan")
            self.printer.println("your account")
            self.printer.println("QR code to")
            self.printer.println("activate your")
            self.printer.println("cart for")
            self.printer.println("your next")
            self.printer.println("order!\n\n")
            self.printer.println("Removed Token:")
            self.printer.doubleHeightOn()
            self.printer.println(self.user_token)
            self.printer.println("\n")
            self.printer.doubleHeightOff()
            self.printer.boldOff()
            self.printer.justify('R')
            self.printer.setSize('M')
            self.printer.inverseOn()
            self.printer.println(" Have a ")
            self.printer.println(" Great ")
            self.printer.println(" Day :) \n")
            self.printer.inverseOff()
            self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
            self.printer.printBMPImage('assets/printer_payment.bmp')
            self.printer.feed(5)

⭐ In the change_layout function:

⭐ Change the layout on the SSD1309 OLED transparent display depending on the ongoing operation.

⭐ Display notifications with the assigned monochrome images.

⭐ Adjust the RGB LED color for each layout differently.

def change_layout(self, activated):
        self.display.clear_buffers()
        if(activated == "home"):
            self.adjust_color([0,1,0])
            self.display.draw_bitmap("assets/cart.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
            self.display.draw_text(48+10, (self.display.height-48) // 2, "Please scan", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "your unique", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "account QR ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "code to    ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "activate   ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "your cart! ", self.light, invert=True)
        elif(activated == "scan"):
            self.adjust_color([0,1,1])
            self.display.draw_bitmap("assets/chatgpt.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
            self.display.draw_text(48+10, (self.display.height-48) // 2, "Please scan", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "a product  ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "barcode to ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "change     ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "the cart   ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "items!     ", self.light, invert=True)
        elif(activated == "register"):
            self.adjust_color([0,0,1])
            self.display.draw_bitmap("assets/registered.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
            self.display.draw_text(48+10, (self.display.height-48) // 2, "Account QR ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "code       ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "registered ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "and your   ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "cart       ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "is ready!  ", self.light, invert=True)        
        elif(activated == "barcode"):
            self.adjust_color([1,0,1])
            self.display.draw_bitmap("assets/barcode.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
            self.display.draw_text(48+10, (self.display.height-48) // 2, "Press:     ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "           ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "Button (A) ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "-> Add     ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "Button (B) ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "-> Remove  ", self.light, invert=True)
        elif(activated == "add"):
            self.adjust_color([0,0,1])
            self.display.draw_bitmap("assets/add.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
            self.display.draw_text(48+10, (self.display.height-48) // 2, "Given      ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "product    ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "added to   ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "your       ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "registered ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "cart!      ", self.light, invert=True)
        elif(activated == "remove"):
            self.adjust_color([1,0,0])
            self.display.draw_bitmap("assets/remove.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
            self.display.draw_text(48+10, (self.display.height-48) // 2, "Given      ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "product    ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "removed    ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "from your  ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "registered ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "cart!      ", self.light, invert=True)
        elif(activated == "payment"):
            self.adjust_color([1,1,1])
            self.display.draw_bitmap("assets/payment.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
            self.display.draw_text(48+10, (self.display.height-48) // 2, "Order      ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "payment    ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "received   ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "and your   ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "cart       ", self.light, invert=True)
            self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "deactivated", self.light, invert=True)            
        self.display.present()
        sleep(1)

⭐ Define the assistant object of the shopping_assistant class.

⭐ Finally, initiate the AIoT shopping assistant (device).

assistant = shopping_assistant()

while True:
    assistant.start()

 

 

 

 

 

 

⭐ Furthermore, I modified the w5300_set_ip function in the wiznet_conf.py file so as to print the Ethernet connection status on the shell.

def w5300_set_ip (self, ip_addr, gw_addr, netmask, dns_svr):
        self.nic.ifconfig((ip_addr, gw_addr, netmask, dns_svr))
        print("W5300 STATIC IP: ", self.nic.ifconfig())
        if(self.nic.isconnected() == False):
            print("\nW5300: Ethernet connection problem!")
        else:
            print("\nW5300: Connected successfully!")
        pyb.delay(500)

 

Step 8: Providing a wholesome e-commerce customer experience with AI-based suggestions

After programming W5300 TOE SHIELD attached to NUCLEO-F439ZI in MicroPython and hosting the e-commerce web application on LattePanda 3 Delta, the AIoT shopping assistant is ready to help customers while physical store shopping.

🛍️📩📲 When the customer provides the required information on the sign-up form on the home page, the e-commerce web application generates a unique account (user) token, creates a unique database table for the new user, and adds the provided information to the MariaDB database.

 

 

🛍️📩📲 After saving new user information to the MariaDB database, the e-commerce application generates an account verification QR code and sends an HTML email to the customer's registered email address via Brevo's Email API, including the account verification QR code.

 

 

 

 

 

🛍️📩📲 Then, the e-commerce application redirects the customer to the user dashboard.

 

🛍️📩📲 If there is no product in the cart, the e-commerce application informs the customer on the user dashboard and the ChatGPT recommendations page.

 

 

🛍️📩📲 The shopping assistant (device) waits for the customer to scan the unique account verification QR code to activate the cart. Also, it turns the RGB LED to blue to inform the user the fact that the cart is not activated yet.

 

 

🛍️📩📲 After scanning the account verification QR code, the shopping assistant registers the obtained user token and turns the RGB LED to green.

 

 

🛍️📩📲 After activating the cart, the shopping assistant waits for the customer to scan a product barcode and turns the RGB LED to cyan.

 

 

🛍️📩📲 When the customer scans a product barcode, the shopping assistant initiates the product menu (interface) to allow the customer to select a command before sending the scanned product barcode to the e-commerce application. Also, it turns the RGB LED to yellow until the customer chooses a command.

 

 

🛍️📩📲 After scanning a product barcode, if the user presses the control button (A), the shopping assistant turns the RGB LED to green and makes an HTTP GET request to the e-commerce application by adding the add command.

🛍️📩📲 Then, the e-commerce application obtains the product information of the given barcode by employing the Open Food Facts JSON API and saves the obtained product information to the user's unique database table.

🛍️📩📲 If the product is already added to the cart, the application increments the given product's cart number by 1.

 

 

🛍️📩📲 After scanning a product barcode, if the user presses the control button (B), the shopping assistant turns the RGB LED to red and makes an HTTP GET request to the e-commerce application by adding the remove command.

🛍️📩📲 Then, the e-commerce application discards the given product from the user's database table.

 

 

🛍️📩📲 According to the received HTTP GET requests from the shopping assistant, the e-commerce application changes the product information in the user's database table.

 

🛍️📩📲 Every 5 seconds, the e-commerce application updates the HTML table rows consisting of the product information of the current cart in the user's database table and the estimated total cart price to inform the user automatically.

🛍️📩📲 After adding products to the cart, the e-commerce application allows the customer to generate ChatGPT-powered suggestions for each product.

🛍️📩📲 When the customer clicks the Ask ChatGPT button, the e-commerce application makes a cURL call to the OpenAI API to generate AI-based recommendations for the given product.

🛍️📩📲 After decoding the API response, the e-commerce application redirects the customer to the ChatGPT recommendations page consisting of the product questions and the provided ChatGPT suggestions.

📌 ChatGPT Recommendations Page for Nutella

➡️ Questions

• What is the nutritional value of Nutella?
• What should I purchase with Nutella?
• Can you teach me a recipe with Nutella?
• How should I serve Nutella?
• Is there a more affordable and healthy option than Nutella?

➡️ Suggestions

• Nutella is a hazelnut spread that primarily consists of sugar, palm oil, hazelnuts, cocoa powder, and skim milk powder. While it is delicious, it is important to note that Nutella is high in calories and sugar. The exact nutritional values can vary depending on the brand and serving size, but on average, a 2-tablespoon (37g) serving of Nutella contains approximately: - Calories: 200 - Fat: 12 grams (including 4 grams of saturated fat) - Carbohydrates: 21 grams (including 20 grams of sugar) - Protein: 2 grams - Fiber: 1 gram - Sodium: 15 milligrams
• Nutella pairs well with many different foods. Here are some popular options to enjoy with Nutella: - Bread or toast: Spread Nutella on a slice of bread or toast for a quick and satisfying snack. - Pancakes or waffles: Use Nutella as a topping for pancakes or waffles, along with some fresh fruits or whipped cream. - Fruit: Dip slices of fruits like strawberries, bananas, or apples into Nutella for a sweet and fruity treat. - Pretzels or crackers: Spread Nutella on pretzels or crackers for a salty-sweet combination. - Ice cream: Drizzle Nutella over a scoop of vanilla ice cream for a decadent dessert.
• Certainly! Here's a simple recipe for Nutella Mug Cake: Ingredients: - 4 tablespoons all-purpose flour - 4 tablespoons Nutella - 3 tablespoons milk - 1/4 teaspoon baking powder - Optional toppings: whipped cream, chopped nuts, or chocolate chips Instructions: 1. In a microwave-safe mug, whisk together the Nutella and milk until smooth. 2. Add the flour and baking powder to the mug and mix until well combined. 3. Microwave the mug on high for approximately 1 minute and 30 seconds, or until the cake has risen and is cooked through. 4. Let the mug cake cool for a few minutes and then add your desired toppings. 5. Enjoy the warm and gooey Nutella mug cake!
• Nutella can be served in various ways. Here are a few suggestions: - Spread it on toast, bagels, or croissants for a quick and delicious breakfast. - Use it as a filling for crepes, pastries, or donuts. - Pair it with fruits like strawberries, bananas, or apples for a sweet and indulgent dip. - Incorporate it into baked goods like cookies, brownies, or cakes for added richness and flavor. - Enjoy it straight from the jar with a spoon (though moderation is key!).
• If you're looking for a more affordable and healthier alternative to Nutella, you can consider making your own hazelnut spread at home. Here's a simple recipe: Ingredients: - 1 cup hazelnuts (roasted and peeled) - 3 tablespoons cocoa powder - 2-3 tablespoons honey or maple syrup (adjust to taste) - 1/4 teaspoon vanilla extract - 1-2 tablespoons vegetable oil (for desired consistency) Instructions: 1. In a food processor, blend the hazelnuts until they turn into a smooth paste. 2. Add the cocoa powder, honey or maple syrup, and vanilla extract to the hazelnut paste and blend until well combined. 3. Gradually add vegetable oil while blending until you achieve the desired consistency. 4. Transfer the homemade hazelnut spread to a jar or container and store in the refrigerator. This homemade version allows you to control the ingredients and adjust the sweetness to your preference, making it a more affordable and potentially healthier option compared to store-bought Nutella.

 

 

 

 

 

🛍️📩📲 When the customer clicks the Checkout button, the e-commerce application redirects the user to the checkout page and transfers the evaluated total cart price (amount).

 

🛍️📩📲 Then, if the customer provides the required credit/debit card information correctly, the e-commerce application places the order, generates a unique payment confirmation QR code, updates the order number in the database, and redirects the customer to the user dashboard.

🛍️📩📲 Since the order number is updated, the e-commerce application is ready to add products to the cart again for the next order.

🛍️📩📲 Then, the application sends an HTML email to the customer's registered email address, including the payment confirmation QR code.

 

 

 

 

🛍️📩📲 When the customer scans the payment confirmation QR code, the shopping assistant confirms the payment, deactivates the cart, turns the RGB LED to white, and prints an invoice via the tiny thermal printer to inform the user.

🛍️📩📲 When the customer clicks the Previous Orders button, the e-commerce application redirects the customer to the previous orders page. The application is capable of displaying all previous orders as separate HTML lists since it saves product information with the corresponding order number.

 

 

 

 

🛍️📩📲 If there are no previous orders, the application notifies the customer on the previous orders page.

 

🛍️📩📲 If the customer clicks the Logout button, the e-commerce application signs out of the user account and redirects the customer to the home page.

 

 

🛍️📩📲 On the home page, the e-commerce application redirects the user to the sign-in page when the customer clicks Already have an account?

🛍️📩📲 If the user provides accurate account information on the sign-in form, the e-commerce application signs into an existing account and redirects the customer to the user dashboard.

🛍️📩📲 While the shopping assistant operates, NUCLEO-F439ZI prints notifications, the Ethernet connection status, the scanned information, and the device status on the shell for debugging.

 

 

 

 

 

 

 

 

 

 

🛍️📩📲 Furthermore, if the customer presses the control button (C), the shopping assistant prints a receipt via the thermal printer to inform the user of the current device status.

As mentioned earlier, I added a small gift for a first-time customer trying the shopping assistant. I printed a Togepi figure with my 3D printer as the gift since it represents good luck and fortune :)

 

 

Videos and Conclusion

Further Discussions

By applying budget-friendly shopping assistants providing AI-based product recommendations, we can achieve to:

🛍️📩📲 bridge the gap between e-commerce and physical store shopping,

🛍️📩📲 enrich the customer experience,

🛍️📩📲 increase the profit margin,

🛍️📩📲 improve local businesses' marketing strategies,

🛍️📩📲 reach the target audience effortlessly.

 

 

 

 

Schematics