This Recommendation describes a crowdsourcing approach for conducting subjective evaluations of speech quality. In comparison to laboratory tests, tests using a crowdsourcing approach rely on participants which are connected via an online platform, and whose task is to evaluate speech quality in their own environment, using their own devices. The current Recommendation gives guidance on the test material, experimental design, and the procedure for conducting listening tests in the crowd. An Annex describes the details of ACR listening quality tests. The method is to be seen as complementary to laboratory-based evaluations which are described in [ITU-T P.800].