Polyfills not an error but eat up precious time

I am trying to speed up the screen display of many of my pages. I know this subject has been mentioned before but I feel that polyfills, although not an error, do eat up precious time. Please correct me if I am wrong because this is not an area I know much about.

My thoughts, based on the screenshot below, are that these polyfills are little snippets of code required for backwards compatibility with IE (again I may be wrong). Because I use Chrome and my client only uses Chrome (they have agreed to standardise on one browser) then IE compatibility is not a requirement.

So, therefore can these polyfills be wrapped in some sort of conditional comments like this (again not an expert so please don’t slate me ) to avoid being called in Chrome when they are not needed.

<!--[if lt IE 9]>
   <script>
      polyfill code
   </script>
<![endif]-->

image624×900 53.6 KB

This is just my layman’s approach to something I don’t really understand