Ability Mat 1 bottleneck... what am I doing wrong?

You're probably skipping your cantina farms/refreshes - I strongly recommend not doing that. Getting your cantina farms done opens you up to farm relic materials without distraction. If you have the crystal income to support it, make sure you're doing 3 cantina refreshes per day - every day. Doesn't matter so much what you're farming - every cantina node drops ability materials.