Automated vulnerability scanning, vulnerability assessment and manual penetration testing are often sold as if they were the same product, but they answer different questions, cost different amounts and satisfy different parts of a regulator's expectations. Buying one when you needed the other is one of the most common, and most expensive, procurement mistakes an institution can make: paying penetration-test prices for an automated scan, or submitting scanner output to a regulator who expected evidence of manual testing. This post sets out what each can and cannot find, and what the National Bank of Rwanda (BNR) expects from the banks and other institutions it supervises.
The difference at a glance
A vulnerability assessment is a building inspector walking through your office and listing everything that is not up to code. A penetration test is a skilled burglar you hire to actually break in, who then shows you exactly how they got in. The assessment is built on automated scanning plus a reviewer who triages the output; the penetration test exploits what it finds and goes looking for what the scanner cannot see.
| Factor | Vulnerability assessment | Penetration test |
|---|---|---|
| Method | Automated scanning and review | Manual testing and tool-assisted exploitation |
| Output | List of known vulnerabilities with severity | Proven exploits, attack chains, business impact |
| Finds business logic flaws? | No | Yes |
| Finds chained vulnerabilities? | Rarely | Yes |
| Speed | Hours to 1 day | Days to weeks |
| Cost | Lower (automated tooling) | Higher (skilled human time; contact us for a quote) |
| Accepted as BNR pentest evidence? | No (not on its own) | Yes (with the vulnerability assessment component) |
| Supports ISO 27001 evidence? | Partially | Yes |
What automated scanners find
Tools such as Nessus, OpenVAS and Qualys check your systems against databases of known CVEs. They identify unpatched software, misconfigurations, weak cipher suites, exposed services and default credentials. A scan fingerprints each host, compares the detected versions against published vulnerability data and produces a list of issues ranked by a severity score based on the Common Vulnerability Scoring System (CVSS). Scanners are fast and repeatable, which makes them well suited to continuous monitoring and patch-management prioritisation.
Strengths: Broad coverage, speed, consistency and a low cost per scan; good for tracking configuration drift over time, for cloud posture reviews, and for clearing up the easy findings before a penetration test. A scanner can sweep hundreds of hosts overnight and flag the unpatched server that a patch-management process missed.
Limitations: Scanners generate false positives that someone still has to triage by hand. They have no model of business logic, so they cannot chain low-severity findings into a critical exploit or test whether User A can reach User B's account by changing an identifier in an API request. A scanner reports what might be vulnerable based on a version banner; it does not confirm what an attacker can do with it. For a regulated bank, that gap is the difference between a tidy report and a defensible security posture.
What manual testing finds
A penetration tester uses tools as a starting point, then applies human reasoning to go deeper. The findings that matter most in banking and fintech environments are almost never the ones a scanner produces. Manual testing uncovers:
- Business logic flaws: a USSD transaction that can be replayed to double-credit a wallet, a transfer limit enforced only on the client side, a loan application that can be submitted with manipulated parameters. These flaws exist in correctly patched, fully up-to-date systems, so a scanner sees nothing wrong. Our USSD security testing guide walks through this class of flaw in detail.
- Authorisation failures: the most critical class of vulnerability in banking applications. Can a regular user access another customer's account data by changing an ID in the API request? Broken Object Level Authorization (BOLA, often called IDOR) is ranked API1 in the OWASP API Security Top 10 (2023) because it is so common and so damaging. A scanner can return a clean report on a mobile banking API, with correct error codes, sound TLS and no known CVEs, while a manual tester changes one parameter and reads every customer's balance and transaction history. Detecting it takes an understanding of which records a given user is meant to see. We cover this class of failure in API security in modern banking.
- Chained attacks: combining an information disclosure finding with a session management weakness and an insecure direct object reference (IDOR) to achieve full account takeover. Each finding alone might be rated low or medium, but together they are critical. A scanner reports them as three unrelated low- or medium-severity items; a tester reports the account-takeover path they add up to.
- Authentication bypasses: weaknesses in token generation, session handling or multi-factor authentication (MFA) that take human analysis to identify and exploit. A
Bearertoken that never expires, a password-reset flow that does not verify ownership, a one-time password (OTP) endpoint with no rate limit: these are the vulnerabilities that turn into headlines.
The output of a penetration test is a documented attack narrative that shows exactly what an attacker could achieve and how they got there. A vulnerability scan demonstrates hygiene. A penetration test demonstrates that someone with an attacker's mindset tried to break in, and documents exactly how far they got.
What BNR requires of banks
BNR requires supervised financial institutions to carry out both components. Regulation N° 50/2022, Article 10, sets the floor: a penetration test at least once a year and vulnerability assessments at least twice a year, with the results filed with BNR. The regulation also expects the testing to be done by qualified professionals and names six qualifying credentials: CISSP, CISM, CISA, CEH, OSCP and LPT, or a similar certification. Our guide to BNR cybersecurity requirements covers the rest of the regulation, including the filing deadlines and how to prepare for an examination.
Automated scanner output on its own does not meet the penetration-testing requirement. An examiner who asks for your vulnerability assessment and penetration testing (VAPT) report will expect evidence of manual testing: attack narratives, proof-of-concept screenshots, and the tester's name and credentials.
Common mistake: paying for an automated scan and calling it a penetration test. Scanner output filed as BNR VAPT evidence is exactly what an examiner is likely to question during an inspection. A genuine penetration test needs a skilled human tester. If you are not sure whether what you are buying is a real pentest, ask to see a sample report with manual testing evidence.
Use both, at different cadences
The right programme uses both, at the cadence each is built for.
Vulnerability assessment: continuous or monthly. Run automated scans on your external-facing systems continuously or monthly as a hygiene baseline. Regulation N° 50/2022 sets a floor of two vulnerability assessments a year for the institutions it covers; monthly scanning keeps you well above it, catches new CVEs and configuration drift between formal engagements, and feeds your patch management process and vulnerability register. If you would rather not run the scanner yourself, IMIZI Monitor watches your external attack surface every day as a managed service.
Penetration testing: annual at minimum. Commission a full manual penetration test at least annually. That is the Article 10 floor for BNR-supervised institutions, and it supplies the testing evidence that ISO 27001 and most other frameworks expect. Testing again after a significant change, such as a new mobile-banking release, a core-banking upgrade or a new third-party integration, is good practice an examiner will recognise, but the regulation itself does not mandate it.
A practical pattern for a BNR-regulated institution: continuous or monthly scanning across all internet-facing assets, an annual full-scope penetration test of the customer-facing applications and supporting APIs, and an additional targeted test after any major change to a payment or authentication flow. Scanning tells you whether your systems are patched; penetration testing tells you whether they can be broken.
Vulnerability assessments cost less because they are largely automated; penetration tests cost more because they need skilled human time. The more useful question is what risk you accept by skipping the penetration test. Our guide to what affects penetration testing cost in Rwanda covers the drivers.
What a full bank VAPT should cover
A VAPT engagement for a financial institution should cover the full attack surface:
- External testing: all internet-facing assets, including web servers, API endpoints, remote access systems and email infrastructure
- Web application testing: the core banking interface, customer portals and admin panels, tested against the OWASP Top 10 and beyond
- API security testing: mobile banking APIs, third-party integrations and internal service APIs, tested for authorisation bypass, BOLA and the rest of the OWASP API Security Top 10
- Mobile application testing: Android and iOS banking apps, covering data storage, certificate pinning, runtime manipulation and deep-link vulnerabilities
- USSD and mobile money testing: session handling, transaction flow manipulation and enumeration attacks on USSD platforms. USSD services handle real financial transactions and are frequently under-tested, so name them explicitly in the scope.
- Internal network testing: simulating an attacker with internal access, testing for lateral movement, privilege escalation and access to critical systems
- Cloud and configuration review: AWS, Azure and hosted environments, tested against benchmark configurations, identity and access policies, and exposed services
If you are comparing providers for this work, our guide to choosing a penetration testing company in Kigali sets out the questions to ask and the red flags to watch for.
How we can help
IMIZI Cyber is a Kigali-based firm providing manual penetration testing for banks, MFIs, fintechs, telecoms, government and healthcare institutions across Africa. Every VAPT engagement combines automated scanning to enumerate known vulnerabilities with in-depth manual penetration testing, and the combined report covers both components with the evidence a BNR examiner expects to see. Testing is led by our lead tester, who holds OSCP (one of the six certifications Article 10 names) and PNPT, and reporting is written for both your board and your engineers. See our penetration testing and security assessments service pages for scope and deliverables, our complete penetration testing in Rwanda guide for what an engagement involves, or contact us to scope an engagement for your environment.