In this webproject as a key goal of the SAE Bachelor program 6GST0XD10x I was building a frontend AND backend based modelcar search application with a react framework called next.js using postgres full-text search and semantic quaring with a vector database and last but not least an EXEL-product catalogue with several cars to feed our database with relevant informations to search for. You can also find access to the project on my developer page: www.svendolin-productions.ch
TYPE | LINK |
---|---|
Check out the official video about this project on YOUTUBE: | WATCH NOW |
Website where the project is going to be implemented in a later period: | www.toycarsaddict.club |
Website where all my projects are staged: | www.svendolin-productions.club |
A) Why building an own search engine?
B) What does the project have to do with model cars?
C) Why not simply install a Wordpress plugin?
D) Why on a separate subpage and not directly on the main page of the car collection?
E) What is clearly a MUST HAVE for my search engine?
F) What is clearly a NICE TO HAVE for my search engine?
G) Why the separation of nice to have and must have?
H) To what extent does the general public benefit from this?
Loading States:
Implementig the Technique of semantic searching with vector based search results:
Full Text Search using a regular serverless relational database:
Advanced Next.js routing patterns:
Caching Search Results:
Product Catalogue as a Seeding Script:
Database:
1) Homepage - Up first it's time to create a basic page without too many details and design elements with following tools:
1.1) React.js is the ideal choice for building a usable user interface as efficiently as possible using components. I had already used other Javascript libraries such as Angular and Vue.js in the past, but I had the most experience with React. Angular was covered in school lessons and used for our PARTUM MEDIA project. That's why I wanted to use something similar.
1.2) Next.js comes into play here as the React Framework, which is based on Node.js and is owned by Vercel. This is ideal if you also want to host or deploy the project on Vercel. There is also a lot on the net about this.
1.3) ShadCN is a UI library for Next.js that I use here to incorporate some standards into the design. For the Bachelor of Science, I can focus on the technical aspects and not waste too much time explaining the UI / UX design.
1.4) Tailwind is a utility-first CSS framework, which provides its users with utility classes and standardized procedures for how the design will behave. So I don't need to spend ages styling individual CSS elements. With the CSS Tailwind Intellisense Extension for Visual Studio Code, I can see and select the utility classes directly in the editor and still have my CSS. But still, Tailwind needs some time to learn and get used to it, so dont judge me too hard if the design is not perfect yet or is buggy here and there. All in all it still should be responsive and user-friendly.
1.5) Lucide.dev is an icon library with a variety of SVG icons that I use here to spice up the design. The elements from ShadCN and Lucide should also behave responsively on the mobile device.
2) Searchbar - Creating an intuitive searchbar which isnt that complicated:
3) Database - Initializing / connecting the application to serverless and vectorized databases:
3.1) PostgreSQL Full Text Search:
3.2) Semantic Quering:
4) Product Catalogue - Creating a product catalogue with a list of model cars + Product Preview
4.1) Faker.js offers me the possibility to embed a certain amount of realistic but fake data into my table of 50 (alphabetical) units, with wrong prices, wrong content etc.. However, the contents are still read correctly and filled into the tables according to their DNA.
4.2) Drizzle.orm allows me to write SQL queries directly in my code, is highly secure and supports PostgreSQL. It is my bridge between the object-oriented search application and the relational database of Neontech PostgreSQL. With the Drizzle Kit Studio I can make adjustments directly.
4.3) Public Folder is the area where my model car pictures are stored. As I photograph a lot of models for a living, I will save them professionally in a suitable size ratio according to the programming and shoot the photos myself in a well-lit photo box.
=> In the Search folder / page.tsx: The first step is the query within our page, this comes from the search parameters that we automatically get passed to the page from next.js. We can simply destructure the search parameters as the props that are passed to the page. The search parameters are objects, each of which has a dynamic key, which is a type of string. The value is a string or a string array or, of course, undefined if the search query is not passed at all (First step is getting the query inside our page, this comes from the search parameters that we get passed automatically be next.js into the page. We can simply destructure the search params as the props that are passed to the page. The search parameters are an Object, each of these has a dynamic key which is a type of string. The Value will be a string or a string array or naturally undefined if the search query is not passed at all)
NODE.JS:
In a NUTSHELL: Node.js is an open-source, cross-platform, back-end JavaScript runtime environment that runs on the V8 engine and executes JavaScript code outside a web browser.
NPM and JSON:
In a NUTSHELL: Node.js offers the Node Package Manager NPM, which allows me to install and manage packages. NPM is also a package manager for JavaScript that is installed with Node.js. It consists of a command line client and an online database of public and paid private packages, called the NPM registry. NPM is the world's largest software registry. It consists of more than 1,000,000 packages (as of 2021). These can also be a collection of CSS classes, a package called Matierial Design for example:
2.1 Open VSC > Open in integrated Terminal to use CLI:
#[!] This would be the correct installation, BUT...:
npx create-next-app@latest #...I had an issue with the latest version of Next.js 15 and shadcn-ui, look at the bottom of this readme! So at the moment you should use the latest 14 version instead by typing:
====>>> npx [email protected]
# Creates an APP with next.js - Then hit YES at using TypeScript / ESLint / Tailwind CSS / etc
npm --version
# Check the npm version and update it if necessary with:
npm install -g npm@latest
2.2 Time to run the DEV Server:
npm run dev // (CTRL + C to stop the server )
What's the difference of RUN DEV and RUN BUILD?
NPM RUN DEV:
NPM RUN BUILD:
[!] Important note: Always check the path: Rightclick on your folder in VSC > Open with integrated Terminal > Run the server from this path (because in my example I initialized .git outside the folder first which was wrong actually. Be sure to initialize git IN the direct folder!)
NEXT.JS and SHADCN:
In a NUTSHELL: It is a React framework and is based on Node.js. Next.JS is owned by Vercel, they also host the projects. Twitch, Tiktok, Ferrari etc. are typical Next.JS users. Hosting via Vercel is as simple as logging in to Vercel and linking to Github so that the project can be uploaded.
Next.js Info: https://kinsta.com/de/wissensdatenbank/next-js
Next.js is the ideal choice for demonstrating an MVP (Minumum Viable Product) to people for testing purposes on December 28. In the end, it will be a more complex and sophisticated web application, called “Full Stack Search Application”.
In addition, Next.JS offers built-in CSS support, which makes my work easier. So I have (theoretically) already created a good basic frontend design in less time and don't waste time on styling or UI/UX design.
3.1 Make things look good with ShadCN (UI library for Next.js, a DEFAULT DESIGN for my project):
More infos: https://ui.shadcn.com/docs
UI Library for React and Next.js:
npx shadcn@latest init
# Installs the latest ShadCN UI package - Then I hit "Default" / "Slate" / "yes"
# It then creates a file called "components.json" where this configuration will be added
3.2 STYLING the Search Bar out of the box with full accessability via ShadCN:
More infos: https://ui.shadcn.com/docs
Part of the UI Library for React and Next.js:
npx shadcn@latest add input
# Installs the fully accessibly input component
# It then creates a file called "input.tsx" which we can use as a basis for our search bar
# It is added at components/ui/input.tsx
TAILWIND:
[+] There is a default configuration that can be easily overwritten with a tailwind.config.js file. = Perfectly customizable.
[+] The auxiliary classes provided by Tailwind save you having to name classes, which makes your work easier
[+] With the IntelliSense Tailwind CSS Extension for Visual Studio Code, users can see and select the utility classes directly in the editor, so I can see the original CSS class names when hovering.
[+] Tailwind CSS is an open source project and can be used free of charge.
LUCIDE.REACT:
npm install lucide-react
NEON.TECH PostgreSQL Database + Fulltext Search:
Why not MYSQL or MongoDB as I usuallly did?
[+] Branching: Free experimentation without effects on the main branch main
[+] Immediate backup of the database. If data is accidentally lost, we can switch to the last good branch.
[+] Simplified integration tests. Developers can run tests in test-specific branches.
[+] Safe testing of automated database migrations in production.
[+] Isolated execution of analysis or machine learning processes.
[+] Connection capability with Drizzle-Kit-Studio to manage everything under the hood!
[!] None of this is possible with conventional database engines. At least not without further ado. Some database engines like SQL Server have snapshots that can actually create instant copies of a database. But snapshots are read-only, and that limits their usefulness. With most database engines, we have to resort to more complicated mechanisms such as backup and restore or replication.
More infos: https://neon.tech/home
Pricing will be set at Free Plan due to testing purposes which can be seen HERE
Neon automatically scales my project's compute resources up or down to meet demand. The recommended settings for my free plan are currently selected at 2vCPU and 8GB RAM max.
Don't forget to copy the unique database URL from the Neon.tech dashboard and paste it into the .env file (as a connection string) which wont be uploaded to Github for security reasons.
6.1 Install the NEON Serverless Driver:
- More infos: https://neon.tech/docs/serverless/serverless-driver
npm install @neondatabase/serverless
# Installs the fully accessibly input component
DRIZZLE.ORM
# Install Note-Postgres Package
npm install drizzle-orm
# Install Note-Postgres Package kit, a studio for managing and interacting to the database
npm install -D drizzle-kit
# m audit fix is intended to automatically upgrade / fix vulnerabilities in npm packages because I had 1 vulnerability
npm audit fix
Time to push our Schema (the datatable) to the database of PostgreSQL by Neon:
# Went through all this: https://orm.drizzle.team/docs/get-started/postgresql-new and directly applied changes to my database using:
npx drizzle-kit push
npm install --save-dev @faker-js/faker
VECTOR / TSX:
With other words: I will implement a semantic search querying to increase the search accuracy by querying semantically related results. More about that: https://upstash.com/docs/vector/overall/whatisvector
9.1 Installment of a vexctor database:
npm install @upstash/vector
9.2 TSX installation:
npm install -D tsx
# And add to the JSON file: "tsx": "tsx", in the scripts section
9.3 Continue with Drizzle (as we knew it from chapter 7.):
npm run drizzle:seed
# And add to the JSON file: "tsx": "tsx", in the scripts section
npx drizzle-kit studio
# And add to the JSON file: "tsx": "tsx", in the scripts section
# After compiling, it's important to copy paste the URL in the browser https://local.drizzle.studio/
9.4 Creating an account:
# https://console.upstash.com/vector/472ec71e-d8f0-4281-bc1c-9dae9212e1bd?teamid=0
OpenAI - OpenAI TypeScript and JavaScript API Library
But what is an API? => The API - also known as the programming interface - therefore enables applications to communicate with each other. The API is not the database or even the server, but the code that regulates the access points for the server and enables communication. This speeds up and simplifies data exchange between different systems many times over.
npm install openai
10.1 Getting used to the OpenAI developer Platform:
Vercel - Publishing projects with Vercel
In a NUTSHELL: Vercel is known for its outstanding deployment and hosting of web applications. With a strong focus on speed and ease of use, Vercel enables developers to publish and manage their projects effortlessly. As with the other applications, a GitHub account is required. More about that here: https://vercel.com/docs
NPM RUN BUILD ERROR => At the moment NPM RUN BUILD is not working due to a bug that we have raised in the issues section. Therefore the project cannot be uploaded to Vercel yet.
if ('generateMetadata' in entry) {
Next.js build worker exited with code: 1 and signal: null
must | Name | Creator | |
---|---|---|---|
✔️ | Prettier - Code formatter | Prettier | link |
✔️ | Tailwind CSS intellisense - CSS tooling. Hover on Tailwind ClassNames to see the regular CSS names | Tailwind Labs | link |
ELEMENT | Library / Component Links |
---|---|
ICONS | LUCIDE |
SEARCH BAR INPUT | SHADCN |
BUTTON | SHADCN |
Feel free to contact me if you've seen something wrong, found some errors or struggled on some mistakes! Always happy to have a clean sheet here! :)
6 issues have been detected, 6 answers have been given, 5 solutions have been made. Check out our ISSUES SECTION for more information: HERE
Questions / Issues | Anwers | Solutions |
---|---|---|
6 | 6 | 5 |