ALTER TABLE DROP COLUMN xxx failed because one or more objects access this column.

I was dropping a column today from a table and I got the following error ALTER TABLE DROP COLUMN xxx failed because one or more objects access this column. The complete message which was coming in SQL Server Query Analyzer message pane was:

Msg 5074, Level 16, State 1, Line 1
The object 'DF_TestTable_RankId' is dependent on column 'RankId'.
Msg 4922, Level 16, State 9, Line 1
ALTER TABLE DROP COLUMN RankId failed because one or more objects access this column.

The reason for this was that I was having default constraint on my column which was not letting me to drop my column from the table.

Solution: 
The solution for this issue was to drop the constraint first and then the column. I did the same as follows:

GO
ALTER TABLE TestTable
DROP CONSTRAINT DF_TestTable_RankId
GO
ALTER TABLE TestTable
DROP COLUMN RankId
GO

and my column was dropped successfully.