Dedicated team for investment portfolio management ecosystem

Dedicated team for investment portfolio management ecosystem

Itransition’s dedicated team of developers and R&D experts created a vendor-agnostic investment portfolio ecosystem featuring custom algorithms and strategy tests for managing billions in investments for thousands of investors.

Challenge

The customer is a large US-based provider of online investment and market research tools for investors and traders. The company creates products for developing equity portfolios, monitoring markets, and selling or buying at the right time.

The customer had many new ideas they wanted to bring to life, so they required a solid technological partner to realize them. They wanted to develop an investment portfolio management system with thousands of active users. The future system had to remain stable and withstand the load of a growing user base without server issues. The potential vendor would also constantly upgrade, extend, and support the product, ensuring system scalability, uninterrupted operation, high security, and 24/7 performance. 

The customer chose Itransition to develop and evolve the product due to our financial software development expertise and experience in providing dedicated teams to cover the whole range of products and services they required.

Solution

After analyzing the customer’s system, we proposed a dedicated team format of collaboration to provide the most fitting products and services and address the customer’s issues in full. Our dedicated team developed an investment portfolio management ecosystem and delivered DevOps, R&D, QA, UX/UI design, and support services. 

Products 

The customer’s ecosystem of products includes web and mobile apps built around market data from leading global stock exchanges, data vendors, and trading experts. 

Core platform

The core platform is a math-based web app that allows individual investors to estimate, track, analyze, and manage their investments. The app features automated investment portfolio management, including risk management, ML-powered price and trend predictions, investment monitoring, and stock behavior analysis. Alerts for best trading strategies help investors select a proprietary trading strategy. Investment portfolios can be imported from online brokers in one click. 

The platform supports almost every US brokerage and most Canadian ones, enabling users to add new brokerages as per their requests. The solution is integrated with multiple stock data service providers. This allows for intraday tracking of equities, funds, indices, and options in the US. The solution also enables users to get end-of-day data on equities in Canada, the UK, Australia, and Germany.

Dashboard
Portfolio distribution

Itransition’s team visualized investments in a dashboard that includes charts, grids, forms, and widgets with calls to action. The dashboard depicts total gain, daily gain, positions in the green and red zones, the performance of all equities, portfolio risk quotient, etc. 

Since widgets became popular with users, Itransition added dashboards to other ecosystem products. Access to the tools is provided according to basic, plus, premium, and pro subscription plans. The frontend components for portfolio management tools create an opportunity to gauge complex data at a glance and make instant data-driven decisions.

Alerts notify users of the best time to close a position and enter opportunities. The system supports alert types and generates millions of transactional emails. It is also optimized to operate flawlessly if the user base grows several times.

Portfolios - Positions

The main platform tools developed by our team enable the following capabilities:

  • analyze a single stock or an entire portfolio to see where user investments stand with an indication of the most optimal stop price
  • analyze the whole portfolio's risks based on each position’s volatility
  • calculate the optimal investment size 
  • create a diversified portfolio from various positions
  • evaluate how diversified their portfolios are across different industries and sectors within the market area
  • visualize price history & other tech indicators
  • create a balanced portfolio 
  • search for a set of investments using specified filters
  • find option trades with a good combination of risk and ROI
  • test custom investing strategies
  • show a calendar of past and future stock market events
  • redistribute risk between existing positions in the user's investment portfolio
  • provide a quick overview of the main stock markets

To keep the financial data up to date, we delivered a set of system services and console utilities that update data several times per hour on a predefined schedule. These services get updates from the data providers the solution is integrated with. We enabled the opportunity to launch them simultaneously on several servers and multi-threaded re-count of statistics and emailing. This allows for faster delivery of alerts to end users when the stock prices go up or down.

Itransition’s team integrated the customer’s ecosystem with multiple data vendors to source, obtain, and store different data types in our system, including market and descriptive data. This allowed us to form a vendor-agnostic historical market database, adding significant value to the customer’s clients.

The customer’s core platform is integrated with the following services:

  • SIX financial information — open-high-low-close prices, corporate actions, bank holidays, delayed intraday prices, and other reference data
  • EDI  — corporate actions
  • SEC API  — companies’ and billionaires’ quarterly reports, financial funds reports
  • Polygon.io   — frictionless access for developers to accurate historical and real-time data
  • EOD Historical Data  — historical prices and fundamental Data API
  • Financial Modeling Prep  —  insider stock market information (news, currencies, and stock prices)
  • IVolatility  — auctions, options, futures, intraday and end-of-day prices, recommendations, surface data
  • Benzinga  — IPO data, earnings surprise, news, markets, premarket prep
  • SendGrid — email API integration for easy delivery 
  • XE (exchange rates) — consolidating end-of-day/intraday stock trading data, as well as information on important corporate events and transformations
  • Plaid and Envestnet | Yodlee — integration with a plethora of brokerages and custodians providing data on portfolios and equities of broker accounts. This integration allows for syncing users’ broker accounts with the customer’s platform. We integrated the solution with both Plaid and Envestnet|Yodlee services to support different broker accounts.
  • Direct Feeds — integrates normalized market data feeds from stock exchanges and trading venues with users’ apps, using algorithmic trading, AI, and ML
  • Custom Salesforce-based CRM system — for managing users’ subscriptions and payments

As the solution depends on integrations with third-party data providers, Itransition’s specialists researched the option of making it vendor- and asset-type agnostic. Previously, to switch to another data provider or work with several providers simultaneously, we had to create an additional system level that would either translate data from any provider into the format we needed or issue our API’s methods to perform certain operations, such as create a portfolio or position. To simplify adding new data providers, we transferred the project's business logic to the platform’s API.

Itransition also developed an investment portfolio management mobile app for the customer. The app features the latest news from financial markets, allowing users to manage investments on the go. It enables creating portfolios to monitor the equities of users’ favorite companies, following company news, and getting an overview of the global financial market state.

Services

Accelerating development

Initially, the project was based on internal SVN servers. Itransition moved the project to Bitbucket, leveraging the benefits of Gitflow and a convenient version control system connected to the code review practice. We moved the project to virtual machines on Ntirety, a hosting provider’s side, to make adding or replacing a server 20x faster. When the customer launches marketing campaigns and expects an increased server load, we can quickly add several new servers to handle the load. 

With the ongoing migration to AWS, some projects are hosted on Windows VMs and some on AWS using RDS. Most databases are located on hosting VMs now. Part of the solution’s apps had already been migrated to Amazon using Amazon ECS and Fargate with Linux containers for launching apps.

With the help of Jenkins, we set up continuous delivery and continuous integration on the project for faster feature delivery and timely user feedback. In addition, we automated routine tasks, like adding new stock symbols to the system (i.e. market capitalization, open, high, low, price-to-earnings ratio, etc.). Previously, end users had to request the addition of the necessary symbols, and developers did it manually. We have developed scripts and integrated them with Jenkins so L1 support specialists can now add symbols without developers' assistance. Our team also automated the delivery of changes to staging environments for the developers to reduce their manual work. Now developers work with updated versions after QA engineers perform automatic tests.

Itransition’s DevOps engineer set up comprehensive Zabbix monitoring with a proactive notification system. In addition to monitoring the servers’ functioning and log errors, we created a large set of business rules that constantly check if the system corresponds with the defined standard. We also use Datadog for performance monitoring app logs, website availability, response time, testing availability, etc. The gathered metrics ensure our apps and infrastructure are observable to achieve secure systems, avoid downtime, and guarantee great user experience.

R&D 

To optimize and diversify risks and increase the profitability of users’ investments, Itransition set up an R&D team consisting of finance analysts and developers that come up with new analytical algorithms for trading. The algorithms optimize risks, track investment profitability, and increase the chances of profit. When developing new tools, Itransition researchers test them on real broker accounts using unique custom algorithms on historical data. This approach has helped us attract new clients interested in making informed decisions when buying and selling stocks. Tests provide the proof of our algorithms’ effectiveness. Clients can benefit from purchase and sale transactions and estimations of how much they could have made if they followed the algorithm. The developed algorithms outperform the S&P 500, and we regularly compare our performance to the index. 

Our R&D team enabled prediction analysis using models that evaluate historical indicators to gauge company stocks and trends. Their AI research achievements are already used internally to automate publishing knowledge base documents, create article summaries, evaluate text sentiment, find keywords, etc. This simplifies the work of publishers and helps structure the project’s knowledge base.

New feature ideas come from many sources, including stakeholders, decision-makers, the product owner, and C-level execs. To organize feature delivery, we filter, evaluate, validate, and prioritize potential features. Backend and frontend developers verify technical feasibility while designers prepare wireframes for the future product. Based on the data, researchers write algorithms to realize these ideas. We present the solution’s new potential functionality to the customer’s decision-makers and analysts and discuss their feasibility. If an idea is greenlit, it’s prioritized depending on feature value, time to market, estimated delivery time, and cost of delay. The new features developed include tools for financial advisers to sell subscriptions to their model portfolios, offering portfolio tracking, custom charts, and widgets, which extends our product to the B2B world’s single ecosystem.

QA & UX

To ensure the solution’s high quality, Itransition’s QA team, comprising manual and automated testing specialists, carried out functional testing, mobile testing (mobile app and mobile layout of the web apps), integration testing, and cross-browser and cross-platform testing. We achieved 98%-100% successful tests after the final release of the platform. The QA team also developed a range of daily autotests, reducing manual testing costs.

To ensure great usability and improve the interaction between users and the product, Itransition’s UX design team implemented the following features:

  • Hints and tips
  • Video descriptions of analytical tools
  • Interactive Getting Started guide

We use WalkMe for interactive user onboarding. With the help of this service, new users get to know the solution’s features by following a step-by-step guide, choosing to go through all the steps or dismiss them. The guide tells users about the product’s components and how to use them. The platform also sends pop-up notifications to users when the new functionality is delivered.

To ensure user-friendliness of the product, we perform regular UX audits, launching features for small audiences, gathering their feedback, and making changes accordingly. This allows us to minimize costs for the customer and perfect the user’s journey.

Support 

To support a large audience, the solution required a customer service system that would allow users to quickly report any issues and get instant assistance from customer support managers. Having researched market offers, Itransition specialists helped to set up the integration between Zendesk and the solution, enabling users to:

  • Easily create issue tickets and track ticket status on the website using a contact form 
  • Use a single space for the support team to instantly view and process user queries and track time spent for problem-solving for further customer service optimization 
  • Access links to FAQs and articles suggested to users as soon as they start typing the issue description, which reduces the number of tickets to process

Since Zendesk integration set up, Itransition continues to provide support services. Our specialists are responsible for configuring and upgrading database server software, evaluating database performance, establishing and maintaining backup solutions, and more. We also gather the parameters for the company’s internal audit and create the affiliate security baseline checklist that includes requirements for server functioning, monitoring and backup organization, access and authentication parameters, and app and code development specifications.

If the monitoring system logs an error, it sends a notification to Itransition’s support team. We provide 24/7 support services, so any problem gets fixed right away. It takes 5-15 minutes to detect common site problems and start working on them. This way, we avoid situations when the customer learns about issues from end users.

Technologies

The system architecture consists of a web part with multiple web servers and database parts. The customer already had an existing web solution written in .NET. Itransition developed the web apps using the ASP.NET Core framework with HTML, CSS, and Vue.js forming the presentation layer. 

The mobile app is Xamarin-based. Microsoft SQL Server, PostgreSQL, and MongoDB are utilized as database management systems. As for the solution’s functionality packages, Highcharts, Highstock, and ECharts are leveraged to enable interactive charts. Using shared components minimizes development time, simplifies support, and enhances performance. 

Previously, there were multiple standalone products hosted on physical servers. Itransition developers provided tech consultancy to the customer to unify all the products into a single platform as an SPA solution hosted in the cloud, using CDN and load balancer to deliver a better customer experience. 

During the project, two hostings supported the customer’s system, but we suggested it wasn’t optimal due to certain limitations. Initially, the solution used hosting.com but gradually we and the customer began considering cloud technologies and planned to migrate to the cloud fully. We considered AWS and Azure as two potential vendors. Since AWS offered a better SLA at the time and our team had successfully set up several AWS services on multiple projects, we selected AWS. Since then, our team started using different AWS services. Because two hosting services were used, we had to deal with challenges such as redundant infrastructure complexity, convoluted communication between vendors, hosting.com infrastructure changes, and bureaucracy. Because of our partnership with Amazon, flexible approach, and suggestion for global infrastructure changes, the customer chose Itransition to perform the full migration to AWS to fully leverage the platform’s capabilities and benefits.

Before the decision to fully migrate to AWS, we carried out the migration gradually, migrating certain system parts using AWS Fargate. Moving away from Jenkins, we opted to use as many native AWS software products as possible not only for the infrastructure we control directly but also for the customer’s side.

Current platform architecture

To reap the benefits of a unified and simplified infrastructure, we automated the full move to AWS and broke it down into tasks performed by multiple teams. 

Migrating to the customer’s solution, we use two methods:

  • as-is migration for easy system parts
  • migration with deeper refactoring for more critical and complex system parts

Itransition’s team chose the best-fitting method for each part to be migrated. Our tech leads planned out the migration of each system part, their quality, and what teams would be responsible for which aspect of the migration, how to migrate databases, add new services, and stimulate all teams to use more AWS services. When migrating to AWS, we changed the current processes, perfecting and optimizing them, changing system architecture, and adding new AWS services to make sure the solution performs better and supports more users. All Itransition teams were involved in the migration.

Itransition drafted the migration roadmap, detailing licensing and server costs. The full migration ensures better system and app performance, a more independent and easy-to-use system, and easy maintenance for both ours and the customer’s Quantitative Analysts & R&D Engineers teams, and other data consumers. Thanks to the AWS fintech startup program, AWS Activate, the customer gets benefits such as AWS credits, tech support, and educational opportunities for optimized performance and risk management.

Itransition has already introduced and implemented dozens of AWS services on the project. To add a section about options to the customer’s solution, Itransition suggested moving to Amazon EC2. The team deployed new subsystems independently, guaranteeing a scalable, adaptable system with no downtime. To store all intraday data and take I/O operations, we suggested TimescaleDB. Our team set up Aurora and MongoDB instances for monitoring intraday prices.

The ELT module runs on AWS Fargate (ECS) used for parsing financial data from third-party data vendors and apps. 

Amazon Comprehend provides natural language processing and machine learning capabilities to: 

  • Discover useful information in unstructured data and text
  • Evaluate and understand the text
  • Mark it as positive, negative, or neutral
  • Add entities to it, such as identifying companies, products, etc.

The module’s goal was to ensure that the processing takes less than a minute. Using several copies of one app, Itransition built multithreaded and multi-instance data processing to set up an allocated cluster with on-demand instances. To scale microservices, we use Amazon SQS, as well as AWS AppSync to create scalable WebSockets for data updates streamed to users in their app.

We refactored the code to achieve maximum efficiency and failover. Amazon S3 is used for keeping static content and long-term database backup storage. To monitor apps and related databases and track logs and metrics generated by services, Itransition suggested using Amazon CloudWatch

The calculation module has the same architecture as the ELT module and performs analysis based on formulas brought by Itransition experts. The module provides data on potential deal profitability, ROI, etc.

To let end users filter their options by more than 80 parameters, Itransition supplied the platform with optional screener functionality. This module is released on PostgreSQL which runs on its own EC2 instance. 

Other AWS services implemented by Itransition are:

  • Amazon Relational Database Service (RDS) — facilitating the setup, operation, and scaling of cloud databases. We currently have eight clusters and large system parts connected with RDS, both using Amazon Aurora and PostgreSQL.
  • Serverless Compute Engine AWS Fargate — setting up our console apps aka microservices.
  • Amazon Secrets — managing safe passwords. 
  • Amazon Simple Queue Service (SQS) — communicating between different system parts.
  • AWS CodeBuild (in test mode) — building and lining up CI/CD pipelines for different subprojects.
  • Amazon Lightsail — launching virtual private server (VPS) instances, containers, storage, and databases, and access levels to them. 
  • Amazon CloudFront — minimizing response time of websites and improving access speed for downloading the content using a content delivery network of proxy servers to cache content. 
  • AWS Database Migration Service (DMS) — preparing for migration.
  • Amazon DynamoDB — running high-performance apps at any scale.
  • Amazon ElastiCache Redis — cashing the most sought-after data to minimize request processing time.
  • Amazon QuickSight — visualizing internal BI insights.
  • Amazon Simple Notification Service (SNS) — delivering critical internal notifications for support engineers.
  • AWS Glue (used by the customer) — streamlining ETL processes and integrating data from multiple sources for analytics.
  • Amazon Kinesis Data Firehose — streaming certain statistics. 
  • AWS Backup simplifying data protection at scale and automatically backing data up. 
  • Amazon SageMaker — allowing for custom ML models’ use in their environment for multiple tasks.

Process

Itransition was responsible for all work from business analysis to development, QA, and DevOps. We had several functional teams working on different project parts, each with their own QAs, PMs, and BAs on board.

Scrum & Kanban

Our teams used different methodologies according to their needs. Scrum was used by backend and frontend teams, with Scrum masters coordinating the activities. The Scrum methodology provided the customer with a clear vision of the project flow, allowing us to gather feedback and adjust the process to any new requirements. Sprints ended with a biweekly release to production and mandatory demos to the customer. They were followed by reviews, overviews of implemented features, and retrospectives to evaluate the progress and gather data for future improvements. Retrospective analysis sessions enabled us to turn feedback into action items prioritized based on issue severity. Due to efficient planning, we had a realistic picture of the project and could forecast bottlenecks, predict the roadmap, and keep all task estimates in sync with it. 

Teams of researchers, integration specialists, DevOps, and mobile developers used Kanban. It ensured process flexibility that allowed them to explore opportunities quickly and provide products and services quickly.

Talent retention

Our dedicated team had high retention and low turnover rates, with all team leads working for the customer for 10+ years. To retain our talent, we clearly described areas of responsibility, enabling specialists to work directly with client representatives, make strategic decisions, participate in collaborations with third-party vendors, and communicate during integrations. We kept our team members motivated by regularly rotating them across different parts of the project to help them apply various aspects of their expertise. Our team members were also encouraged to show initiative, do research, and explore interesting directions in their work. 

Thanks to this approach, Itransition gained domain expertise, project knowledge, and integration experience that allowed us to be strong technological partners of the customer. This includes providing autonomous decision-making and solution optimization services and choosing short and long-term goals and strategies to achieve them. 

Knowledge sharing

Together with the customer, we prepared a wiki space with a knowledge base on project architecture, detailed project documentation, coding guidelines, know-how tips, and workarounds for recently resolved issues. Itransition also organized developer channels to exchange new ideas and the latest news on relevant topics. R&D experts as the main domain experts together with subject matter experts prepared and conducted quarterly domain-focused educational presentations. We held brainstorming sessions and trading contests for developers on successfully trading stocks, sharing the gathered expertise with end users to widen their skills.

To manage staffing rate risks and enable constant scale-up and steady growth, we developed an onboarding program in line with the customer’s needs. The established knowledge-sharing process includes an onboarding guide with links to informational resources about the investment domain. It helps newcomers quickly grasp project details and find answers to project-related questions. Even if a team changes on any side, knowledge retention ensures all project data remains intact. 

To help scale up projects according to the customer’s changing needs, Itransition introduced an internship and mentorship program. Newcomers initially worked with a mentor who shares internship tutorials with them, introduces them to the project and the team, and describes the existing processes. Interns performed more basic tasks, allowing experts to concentrate on more challenging project aspects. Interns who excel at domain and project skills, with the approval of the customer, were involved in the project fully. We strived for all to serve as mentors and not only serve as educational experts but also give feedback on the process to make it more efficient.  We also performed regular code reviews, which enabled knowledge sharing and helped newcomers quickly get an overview of the existing code.

Results

Itransition delivered an investment portfolio tracking and management ecosystem by leveraging analytical tools, actual stock data monitoring, and an alert system. During our collaboration with the customer, we grew as a domain-savvy and loyal technology partner, working to achieve mutual benefits as part of a dedicated team that features highly competent developers and R&D experts. Since our developers managed to implement new ideas from the R&D department quickly, it allowed Itransition and the customer to attract billions in investments for thousands of investors.