Many shops use eXtreme Programming (XP). They skip planning and focus on building at high throughput. The result normally costs 10x more than what the client expected. Sometimes this happens because the client thought that one feature would be simple, but if he knew it was that complex, both parties would be able to haggle the scope. That's why we use a boring architecture that is easy to estimate based on objective parameters such as the number of columns or fields you need for each page. Plus, building without planning, not even a sprint beforehand, is like laying bricks without a blueprint, which normally leads to avoidable mistakes and requires reworking everything built upon the broken bit.
Sometimes this higher cost happens because of the overexcitement to build it more technologically robust than the business needs or to overengineer it for the hypothetical future. Some shops will opt for the state of art microservices/modular (recommended for mission critical high traffic software such as fraud detection or e-commerce), or the sophisticated clean architecture (recommended for complex planning system such as school time table), when the flat MVC architecture (recommended for most of ERP, CRM and BI systems) would be enough. Plus, some shops will add too many resume-shining components to the architecture, such as multiple databases (it only makes sense for huge projects), creating data inconsistencies and a lot of inglorious work to try to keep everything synchronized.
The Web was built through broad debate among the brightest minds and several companies. Decades of experience formed the base of it, but it would be always open to be extent. Many shops will recommend customizing it the most to create a Facebook-like experience. This involves replacing the Browser native frontend with a custom made JavaScript frontend. A decision alone that is prone to skyrocket the costs of the project. One obvious reason is that you have to build, test and fix bugs of features such as data requesting and screen updating, things the browser is capable of doing by default. Plus, every time a feature change you'll have rework the backend and the frontend which normally translates to double hiring developers.
Another unexpected reason is that highly customized frontends break with basic Web 1.0 principles that were thought to maintain everything simple. The MVC architecture, that we already mention, fits perfectly over these standards: the site offers options (such as links or forms), the user makes a small decision, the site offers new options based on the user's prior decision, and so on. The navigation is incremental, it is easy to build, to maintain, and to use. To break with these standards means poor integration with SEO and URL sharing, too many buttons in one screen to overwhelm the user and to not become mobile-friendly, more work of creating two interfaces for API and Web, a much complex server architecture to handle multiple decisions at once (payload), and a great deal of complexity to make it scale.
The #1 cost in software development is the so-called technical debt, which boils down to bugs hard to fix. Most shops will build your software using a procedural (or object-oriented procedural) language. A procedure is a very powerful tool that enables you to do anything, especially to change the value stored in memory. This means you can test the software a thousand times and it will work, but someday the memory will change just to the right configuration that makes it spoil. Sometimes it will be even hard to discover for which memory configuration that algorithm does not work, let alone what is the issue with the algorithm. This means a high cost to guarantee dependability through tests. Some shops will even create another software to test the original software to save costs by automatizing some of these tests.
Instead of trying to stabilize something unstable, we prefer to build something stable by design. We use functional or even descriptive languages instead of procedural whenever possible. One is able to build a software that is less error-prone by only deciding to work with a less powerful language, such a language with the discipline of never changing the memory. Remember your Math classes, when you defined , it continued to be 1 no matter what happens. If you find out somewhere else, you got an obvious contradiction and you knew that something went sour. This is the kind of immutability that makes our software more reliable at detail level. Technology is just like a house of cards, to build over an unstable base is the path to collapse when it becomes bigger.
Most shops will create highly customized solutions, no matter if they built it within a framework or not. Because they are made from scratch, they need to be tested and fixed over the years, the technical debt we already talked about. The same way they will build custom JavaScript instead of using the browser's native resources, they will build custom backend instead of using the database's native resources, they will create custom integrations instead of using an integration tool, and they will codify custom business rules instead of using a rule engine. Even worse, they will create their own abstractions to avoid duplicated code that shouldn't even exist in the first place, and then it will be cumbersome to read a code full of undocumented abstractions that nobody else knows, not even the authors after three months.
Die-hard OpenSource tools are built and maintained especially by volunteers that also use them in their job. This means they need to be simple to build, to maintain and to use. They are designed to just push up the decisions from a free-form context to a more structured specific domain, where the configuration is preferred over customization. The duty of learning how these tools work and how to configure them easily pays off with dependability and standardization. But, in technology there is no one-size-fits-all solution, each configuration exists because it represents a trade-off that must be taken into account. Many shops will tell you that the low level details matter no more because of vibe coding, huge mistake. Ignoring the trade-offs will cost you more components, more complexity, more infrastructure and sometimes they won't work for your concrete scenario.
Italian School BI platform
This school had some of its reports running through Excel until the volume of data increased so much that they couldn't handle anymore. We developed the system from the beginning, starting with the data model optimized to OLAP for 85 million rows, going all the way to the web interface, SSO and integration with other educational resources. Plus, one business requirement was the ability for the teacher to create ad-hoc reports based on the materials he uses in class. It was a pleasure to discuss the custom business requirements and provide insightful technical experience to achieve these goals seamlessly.
US Shopify BI platform
This startup built a BI platform for Shopify shops to find trends in their sales data through complex analysis such as CAC/LTV and Cohort. When we joined this on-going project, the server supported only one user at a time because of the data intensive operations for hundreds of stores. We found out some bottlenecks regarding the cloud usage and the ingestion process and fixed them. The main challenge though was to optimize the database. Because we didn't participate in the data model phase and the company had a small budget to handle big fixes, we had to find low hanging fruits of optimization that could provide enough improvement.
US Real Estate webscrapper
One realtor needed to ingest foreclosure data from multiple counties and to match it with his proprietary database to find good deals. Beyond the database modeling challenge, this project also involved use of tools to convert images from PDF into text. The original solution proposed by the client was a general-purpose AI which would be very expensive. We recommended using a tailored AI for OCR, which reduced 100x the cost per PDF ingested. Another challenge was to parse the legal description of the properties into specific fields that could be matched in the database.