Oddly enough, your focus could be wavering. There is a mental component to sex. If you're not thinking about what you are doing, it will take longer to come. Does your mind ever waver during sex?