It's not 100%, but it's a quick check. If it rubs off it's almost certainly fungal, so I wouldn't bother treating for bacterial. If it doesn't rub off, it still might be fungal - but in my experience, it's more likely bacterial.
And don't be too quick to rule out pests - there are some that are...