Reinforcement learning example when the action is a matrix

I am working on solving a problem with reinforcement learning which has to find the optimal matrix that maximize the reward. I am not able to see how I can formulate this problem as I have practiced example with simple actions like UP DOWN LEFT RIGHT. My actions are matrix of size (Nt, Nr) where Nt and Nr can be 64. I have tried to convert it to one dimension and use DDQN but when I run Q[action], I get an error because a list can only be indexed with an integer.

Please help me to find at least what I need to explore first. Do I need to use A2C networks?