Hand-Cipher for fun and blackouts

Oh noes, captain! We have to deliver this urgent message, and our enemy has mysteriously managed to blow up all our computers!
Fear no more, citizens. For I bring you this (hopefully, and at least for your evildoer neighbours) hard-to-crack
handcipher. It takes some time to encrypt, but well. What can you do without precious electricity!
Heres an image (say nothing: i know it's beautiful):



The word we want to encrypt is: "goose". This is because I don't have much time left to write this.
we pick two passwords: one of letters (the directions, as seen above), and one of numbers. About 3 units long should be enough.
732 / adh
Now, for the first letter, G, we do this:
Look where it is seen in the letter matrix (by the way: change it as you like it...), take the first direction letter "a", and the first number "7". Move 7 Spaces according to the direction letter. If you hit the edge, start at the other side. "a" means: up. 7 means: seven.
The G now is a Y! Proceed like this until you have applied the whole password on the letter.
The final product would be an H.
Now, with the next letter, do exactly the same, however: Shift the passwords. You now start with the second letter/number of the password, and go on to the next, until you hit the last one, and then apply the alphanumerics (awesome word) you have skipped like this.
When you start encrypting the third letter, the part of the password you would start with is: 2 and h.

For deciphering this, pass the two passwords and let the poor receiver inverse the whole procedure.

Questions? I shall answer them tomorrow. Also, I would love for someone with expertise to rate this system.
May the alphanumerics be with you (or not. hah.)