Case study/Security
Multi-factor authentication platform: Data architecture refactoring
Solus Connect is an innovative multi-factor authentication platform that uses 3D facial authentication and machine learning to track down fraudulent behavior.
Solus Connect's multi-factor authentication platform ran on a single database that was growing faster than the database technology of the time could handle. unicrew re-architected that data layer, partitioning it horizontally with sharding so load could spread across multiple database nodes and the platform could carry much larger datasets and transaction volumes.
- Client
- Solus Connect
- Focus
- Security, Data Management
- Market
- Asia
- Engagement
- Data architecture refactoring
- Stack
ASP.NETWeb API
Bootstrap
MS SQL
Project overview
Our client is a breakthrough startup headquartered in Singapore, working in secure user authentication with biometric and machine learning technologies. Its solution helps enterprises worldwide address two key digital challenges: online fraud and unauthorized access to digital assets.
Solus Connect features an internal Risk Scoring Module that uses AI-based prediction algorithms to detect fraudulent behavior among normal authentication attempts. At its core, the platform leverages predictive machine learning that weighs three main inputs: the user’s device, facial 3D attributes, and user authentication behavior.
Challenge
Like any machine learning platform, Solus Connect processed an impressive number of datasets every second, all requested from a single database that was growing faster than the database technology could handle at the time. During a period of very rapid growth, with the customer base expanding daily, the increased load on the platform created a second challenge: performance and horizontal scaling.
The team was entrusted with a mission to design and implement a solution that would significantly improve the platform’s performance and ensure the new database architecture could facilitate an indefinite number of datasets.
Solution
In this kind of software, the main bottleneck for performance optimization and horizontal scaling sits deep inside the database architecture design. There was no easy tweak, so we had to make serious design decisions to meet our client’s expectations.
After a quick R&D phase, the team decided to move forward with a horizontal partition of the data in the existing database, redesigning it using the sharding method, which splits a single logical dataset into multiple shards (or databases).
Result
Sharding spread the load across multiple database nodes instead of concentrating it on one, and that is what let the platform support much larger datasets and transaction volumes than the single-database design could carry.
The refactoring sat inside a longer engagement. unicrew, then trading as Artelogic, was building the platform to the client’s requirements alongside it, which is the name Matthew Ainscow uses in his Clutch review.
Scaling the data layer under a machine learning workload is what our data engineering team gets called in for, and clients describe those projects in their own words on our client reviews page. For a different shape of security engagement, there is the iOS and Android apps for a building-security company.
What the client says
We scoped out the workflow for the platform. Using this foundation, Artelogic’s executing our requirements and developing the platform. Artelogic executed flawlessly according to our plan thus far. Their work ethic is impressive.
05/Technologies
Technologies on this project
06/Quick answers
The questions behind the project
What did unicrew do for the Solus Connect platform?
unicrew re-architected the data layer behind the platform. The whole dataset was being served from a single database that was growing faster than the database technology of the time could handle, so after a short R&D phase the team partitioned it horizontally, using sharding to split one logical dataset across multiple shards. Load could then spread across multiple database nodes, which is what let the platform carry much larger datasets and transaction volumes.
When does a scaling problem need a data-architecture change rather than tuning?
When the limit is in the shape of the data rather than in how it is queried. Indexes, caching and query rewrites raise the ceiling; they do not remove it. The signal to watch is whether more hardware still helps: once vertical scaling has stopped paying and the read and write paths cannot be split any further, the remaining option is to partition the data, and that is a design decision rather than a tuning pass. It is worth reaching for deliberately, because re-partitioning a dataset that is already live is a great deal more work than tuning one.
What experience does unicrew have with data architecture on security platforms?
Security and fraud-detection products tend to be read-heavy and latency-sensitive at the same time, because a scoring decision sits in the middle of someone's login. That is the shape of problem our data engineering team works on: partitioning schemes, spreading load across nodes, and the data modelling behind machine learning workloads that score traffic as it arrives. The engagement here was that work, on an authentication platform.