I have a matrix in MATLAB. I want to check the 4-connected neighbours (left, right, top, bottom) for every element. If the current element is less than any of the neighbours then we set it to zero otherwise it will keep its value. It can easily be done with loop, but it is very expensive as I have thousands of these matrices.

You might recognize it as nonmaxima suppression after edge detection. 

Answers: 

If you have the image processing toolbox, you can do this with a morpological dilation to find local maxima and suppress all other elements. 

array = magic(6); %# make some data

msk = [0 1 0;1 0 1;0 1 0]; %# make a 4-neighbour mask

%# dilation will replace the center pixel with the 
%# maximum of its neighbors
maxNeighbour = imdilate(array,msk);

%# set pix to zero if less than neighbors
array(array<maxNeighbour) = 0;

array =
    35     0     0    26     0     0
     0    32     0     0     0    25
    31     0     0     0    27     0
     0     0     0     0     0     0
    30     0    34     0     0    16
     0    36     0     0    18     0

edited to use the same data as @gnovice, and to fix the code