I should have elaborated my reply a bit more... I would wait until the cutting has formed some roots, which for my climate conditions and way of working is typically after about 4-6 weeks (for peppers). I always remove most of the leaves, especially the older and/or larger ones.
If working with...