Unveiling Vulnerabilities in Bitcoin's Misbehavior-Score Mechanism: Attack and Defense

The Bitcoin network is susceptible to various attacks due to its openness, decentralization, and plaintext connections. Bitcoin created a misbehavior-score mechanism for monitoring and tracking peer misconduct. In this paper, we uncover several vulnerabilities of this mechanism, leading to potential Bitcoin-Message-based Denial-of-Service (BitMsg-DoS) attacks on Bitcoin nodes and Slander attacks by maligning innocent nodes. We prototype these attacks for our experiments by testing real nodes connected to the Bitcoin main network (while we do not exfiltrate our attacks to the real-world main network). The experimental results show that the attacks exert varying degrees of impact on mining and non-mining nodes, notably reducing mining rates by up to half for affected mining nodes and decreasing the synchronization speed of blocks for non-mining nodes. To address these drawbacks, this study proposes corresponding countermeasures targeting the identified vulnerabilities in the misbehavior-score mechanism. Furthermore, we explore the Peer-to-Peer (P2P) encrypted transport protocol with experimental support in the latest Bitcoin Core 26.0, but find it insufficient in mitigating the Slander attacks.