A pastor is a man that is called by God to shepherd the flock and to preach the
Word of the Lord to them. The church will only submit to the pastor to the degree that the pastor is submitted to Jesus.
While the pastor is the spiritual leader of the church, he is not Jesus. Your pastor has never walked on water. Your pastor has never risen from the dead. Your pastor does not (should not) claim to be a deity.
Some churches wrongly worship the pastor. They worship the messenger instead of the one that the message is about. The pastor should be pointing the church to follow Jesus; not himself. Anything that you place before Jesus in your life, is an idol.
Any man that tries to take any Scripture that pertains to Jesus and apply it to himself is a false prophet.
If God calls the pastor to leave the church and go pastor a church somewhere else and half the congregation goes with him, those people most likely hold the man in higher esteem than he should be held. When God moves a pastor to another church, he does not always call people to move with him. The man might be a good communicator. He might be a good leader. He may be inspiring. He may have discipled you. He may have dedicated your baby or officiated your wedding ceremony, but he is not Jesus.
Any church that is about its own glory or the pastor’s glory is in danger of having their lampstand removed from its place. If the pastor is not about God’s glory, it is highly likely that the church will not be either.
What you can and should do is pray for your pastor. He is held responsible for how he communicates the whole counsel of God to you. He prays for you. He wrestles with certain texts and desires to continually point you to Jesus. Pray for him.
God will hold him to a higher accountability for how he shepherded the flock that God gave him.
He needs all the prayer he can get.
Remember, the pastor is to point you to Jesus. He is not Jesus.