The short answer to the question is YES!
The reason is simple. At least a third of your website visits will be via a smartphone. If your site is hard to navigate on a phone, you may lose potential customers.
In the bad old days, it was necessary to have one website for the desktop clients and another one for mobile devices. That is no longer true. By using a responsive website, you can have a single website that delivers content and navigability on all platforms, including smartphones.
If you are using WordPress as your website platform you will find many available responsive themes. With a responsive theme your website will be easily navigable on all formats (phone, tablet, laptop, desktop).