Book a Call
Get a Quote

Mastering Schema Validation in Retool: Ensuring Flawless Data Integrity

Nhan Hoang
September 28, 2024
20 min read
Mastering Schema Validation in Retool: Ensuring Flawless Data Integrity

Maintaining data integrity is vital for any application, and schema validation in Retool makes it easy to enforce data rules. This blog offers a clear guide to mastering schema validation, ensuring your applications are consistent, reliable, and error-free.

What is Schema-Based Validation?

Schema-based validation is the practice of defining a set of rules that govern the structure and content of data inputs. A schema serves as a blueprint for validation, outlining what is required for each field, permissible values, and interdependencies among fields. This approach centralizes validation logic, making it easier to manage complex data structures and enforce business rules consistently.

Key Benefits:

  • Consistency: Ensures uniform validation rules across various components.
  • Maintainability: Simplifies updates to validation logic by centralizing rules.
  • Clarity: Provides a clear overview of validation requirements, making it easier for developers and stakeholders to understand.

Why We Need Schema-Based Validation

Schema-based validation is essential for several reasons:

  1. Data Integrity: It helps prevent invalid or malicious data from entering your application, which can lead to errors or security vulnerabilities.
  2. User Experience: Clear and concise validation messages improve user interaction, guiding them to provide the correct input.
  3. Complex Validation: Traditional validation methods often fall short when it comes to validating nested or interdependent fields. Schema-based validation can handle these complexities gracefully.

Why We Need It in Retool

Retool is a powerful platform for building internal tools, which means we work with form all the time, schema-based validation empowers Retool users to:

  • Validate complex data structures directly within the platform.
  • Implement business logic that governs user input effectively.
  • Centralize and reuse validation schemas across multiple forms or components.

How to Use the Validation Library

Installation

In retool, we can add a library (that I built it myself while working on several project that involve a lot of form and data consistency - called Validation) using this URL:

https://raw.githubusercontent.com/hctxnhan/Validation/refs/heads/main/dist/validation.umd.min.js

How to install Validation Library

To implement schema-based validation in Retool, follow these steps:

  1. Define a Schema: Create a schema that outlines validation rules for each field.
  2. Create a transformer to handle your validation logic, incorporating the defined schema.
  3. Call the validation function when user inputs are submitted or changed, passing the input data and schema.
  4. Display error messages based on validation results, enhancing user interaction.
  5. Handle Errors: Return and display error messages to users based on validation results.

Example Schema

Here’s an example schema that validates user input for a registration form:

1const exampleSchema = (data) => ({
2  name: [
3    {
4      validate: [
5        validation.STRING.isRequired,
6        validation.STRING.type,
7        validation.STRING.minLength(3),
8        validation.STRING.maxLength(20),
9      ],
10    },
11  ],
12  age: [
13    {
14      validate: [
15        validation.NUMBER.isRequired,
16        validation.NUMBER.type,
17        validation.NUMBER.minValue(18),
18      ],
19    },
20  ],
21  birthDate: [
22    {
23      validate: [
24        validation.DATE.isRequired,
25        validation.DATE.type,
26        validation.DATE.isPast,
27      ],
28    },
29  ],
30  isMale: [{ validate: [validation.BOOLEAN.isRequired, validation.BOOLEAN.type] }],
31  idNumber: [
32    {
33      validate: [
34        validation.STRING.type,
35        validation.STRING.isPattern(/^[0-9]{10}$/),
36      ],
37      when: [validation.$OTHER("age", validation.NUMBER.minValue(18))],
38    },
39    {
40      validate: [validation.STRING.isEmpty],
41      whenNot: [validation.$OTHER("age", validation.NUMBER.minValue(18))],
42    },
43  ],
44  email: [
45    {
46      validate: [
47        validation.STRING.isRequired,
48        validation.STRING.type,
49        validation.STRING.isEmail,
50      ],
51    },
52  ],
53  url: [{ validate: [validation.STRING.type, validation.STRING.isUrl] }],
54  assets: {
55    $beforeAllWhen: [validation.$OTHER("age", validation.NUMBER.minValue(18))],
56    cars: [
57      {
58        validate: [
59          validation.ARRAY.isRequired,
60          validation.ARRAY.type,
61          validation.ARRAY.minLength(1),
62          validation.ENUM.oneOf(["toyota", "honda", "ford"]),
63        ],
64      },
65    ],
66    carTax: [
67      {
68        validate: [
69          validation.NUMBER.isRequired,
70          validation.NUMBER.type,
71          validation.NUMBER.minValue(100),
72          validation.NUMBER.isInteger,
73          validation.NUMBER.isPositive,
74        ],
75        when: [
76          validation.$OTHER(
77            "assets.cars",
78            validation.ARRAY.type,
79            validation.ARRAY.minLength(1),
80          ),
81        ],
82      },
83    ],
84    max: [
85      {
86        validate: [
87          validation.NUMBER.isRequired,
88          validation.NUMBER.type,
89          validation.NUMBER.isInteger,
90          validation.NUMBER.isPositive,
91        ],
92        when: [validation.$OTHER("assets.min", validation.NUMBER.type)],
93      },
94    ],
95    min: [
96      {
97        validate: [
98          validation.NUMBER.isRequired,
99          validation.NUMBER.type,
100          validation.NUMBER.minValue(0),
101          validation.NUMBER.maxValue(lodashGet(data, "assets.max")).mgs(
102            "Can not be more than max",
103          ),
104          validation.NUMBER.isInteger,
105          validation.NUMBER.isPositive,
106        ],
107        when: [validation.$OTHER("assets.max", validation.NUMBER.type)],
108      },
109    ],
110  },
111  numOfFollowers: [
112    {
113      $beforeAllWhen: validation.$OTHER(
114        "url",
115        validation.STRING.type,
116        validation.STRING.isUrl,
117      ),
118    },
119    {
120      validate: [
121        validation.NUMBER.isRequired,
122        validation.NUMBER.type,
123        validation.NUMBER.between(10000, 500000),
124      ],
125      when: [validation.$OTHER("age", validation.NUMBER.minValue(18))],
126    },
127    {
128      validate: [
129        validation.NUMBER.isRequired,
130        validation.NUMBER.type,
131        validation.NUMBER.between(5000, 10000),
132      ],
133      when: [validation.$OTHER("age", validation.NUMBER.maxValue(17))],
134    },
135  ],
136  $refine: [
137    {
138      validate: [
139        validation.AGGREGATE.all([
140          validation.$OTHER("age", validation.NUMBER.lt(18)),
141          validation.$OTHER("idNumber", validation.STRING.isEmpty),
142        ]).mgs("idNumber cant be specified for under 18"),
143      ],
144      path: ["idNumber"],
145    },
146  ],
147});

I have prepared an application in Retool to demonstrate this example:

Application in Retool

And a transformer to perform validation on the form data:

Transformer to perform validation

For each and every input in the form, instead of define the validation logic individually in the component, now we have the validation errors prepared and centralized in the transformer result, we can use formDataKey to get the error of the component and put it into Custom rule in Validation section.

Have the validation errors prepared and centralized

That’s still create repetitive in the process, but that less error prone than when we specify the logic in here, when something changes, we don’t need to go and find it everywhere and update it.

Let’s try changing data in the form to see what happen to it.

How to Extend the Validation Rule

The validation library is designed to be extensible. You can introduce new validation rules by defining custom logic within your schema. For instance, you can create a "between" rule to ensure a value falls within a specified range:

validation.CUSTOM({
  validate: (value) => value >= 10000 && value <= userSpecifyMaxInput.value,
  message: `${field} must be between 10000 and ${userSpecifyMaxInput.value}.`,
});

How Extensible and Flexible the Validation Rules Are

The validation rules can be tailored to suit specific requirements. You can

  • Nest rules.
  • Inverted any validation rule.
  • Customize the validation message.
  • Create conditional validations based on other field values, or
  • Introduce entirely new rules as your application grows.

This flexibility ensures that the validation logic can evolve alongside your business needs.

Example of Flexibility

In the previous schema, the idNumber field's validation changes based on the age field. This interdependence showcases how validation can adapt based on input context.

Using the Schema for Validation

To validate an object against the schema, use the following:

const errors = validateObject(exampleSchema(data), formData);
if (Object.keys(errors).length) {
  console.log('Validation failed:', errors);
}

Conclusion

Schema-based validation is vital for ensuring data integrity and improving user experience in Retool applications. By defining clear validation rules and leveraging the extensibility of the validation library, developers can create robust, reliable forms. For further details, please refer to the Validation | GitHub repository for this library.

Implementing schema validation can greatly enhance your application's resilience against invalid data and streamline the user input process, making it an essential part of any Retool project.

Get in Touch

Ready to bring your dashboard vision to life? Contact Retoolers today, and let us help you create a powerful, intuitive dashboard that meets your exact requirements.

Nhan Hoang
Retool Developer