First you need to make sure you have the latest version of wget, some distros are still being released with older versions of wget which has some bugs regarding mirroring functionality. currently the latest version is 1.13.4, so if you don’t have the latest version, you can download and build it from following link :
ftp://ftp.gnu.org/gnu/wget/
after building the wget, make sure the latest version is being used :
wget -V
output :
GNU Wget 1.13.4 built on linux-gnu.
+digest +https +ipv6 -iri +large-file +nls -ntlm +opie +ssl/gnutls
Wgetrc:
/usr/local/etc/wgetrc (system)
Locale: /usr/local/share/locale
Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc"
-DLOCALEDIR="/usr/local/share/locale" -I. -I../lib -I../lib -O2
-Wall
Link: gcc -O2 -Wall -lgnutls -lgcrypt -lgpg-error -lz -lrt ftp-opie.o
gnutls.o ../lib/libgnu.a
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.
OK, you are good to go now, just execute the following command and relax 🙂
wget -mkp -e robots=off http://site