Website performance is an important goal no matter the platform or CMS alternative that you are using. Studies have consistently shown that the longer it takes for your website to load the more likely it is that your visitors will "bounce" off of the page and move on to view something else.
Within the Drupal community, there have been discussions and concerns over the last couple years about the performance of the upcoming Drupal 8 release. Newer technologies being added to the Drupal 8 core such as "BigPipe", will likely have great benefits to overall page loading speed.
However, for people still using Drupal 7, and assuredly many will for some time to come, it is important to realize that even though the core Drupal 7 setup already has a lot of optimizations and tools for boosting performance like the built-in caching system. There are several contributed modules for Drupal 7 that one can install that will greatly enhance the ability of the core caching system to reduce page rendering time for regular Drupal website visitors.
Advanced CSS/JS Aggregation
The Advanced CSS/JS Aggregation provides a wide variety of enhancements to the caching process to signifcantly boost front-end performance. This includes the automatic generation of aggregate files if they aren't present, and stampede protection to provide locking in the case of multiple requests for the same file. Theme files are combined using media queries for better grouping, and compression and I/O is better processed to reduce the amount of files to be loading.
This is just one of those modules that probably everyone should install and could potentially get some benefit from.
Block Cache Alter
One deficiency in Drupal 7 core is the lack of more varied control over how blocks are cached. Primarily you get an option within the "performance" settings page to either cache all blocks or not. It's possible though that you may want to have more granular control over which blocks get cached or not. This contributed module will help you with that. Additionally you might consider using the Ajax Blocks (https://www.drupal.org/project/ajaxblocks) module to facilitate caching of the overall page, but have some blocks with dynamic content load through an Ajax operation.
Panels Hash Cache
Drupal Panels are an incredibly useful tool for generating more interesting and varied display layouts for your content. However, their complexity and the dynamic nature of their contexts don't lend them to work well with Drupal's core caching. This contributed module provides a hash-based cache for Panels supports caching based on any combination of contexts, arguments, roles, etc.
Entity Cache Loader
Caches help with performance but at some point they need to expire in order to be effective as a part of your website's content presentation. When caches expire, they then need to be reloaded, and if the caches all expire at a similar time the loading process may cause undue performance loss for a subset of your visitors. Cache warming is a process of pre-loading content to be cached beforehand instead of at the time of actual demand. Ensuring all of your content is cached at all times will greatly help boost the performance of your Drupal website.
Views Content Cache
Along with having your content cached as much as possible, sometimes you need to have your cache refreshed in order to reflect changes in your content. The Views Content Cache module helps to maintain the accuracy of your cached Views pages by letting them "monitor" specified content types for changes, and thus expiring and renewing the caches for those Views to reflect the updated content.