Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
 
     
   Compare ISP   Rate your ISP
 
Please login or register.

Login with username, password and session length
Advanced search  

News:

Pages: [1] 2 3 4

Author Topic: DNS records or htaccess help please  (Read 2503 times)

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 31543
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
DNS records or htaccess help please
« on: April 27, 2018, 10:07:21 PM »

Can anyone help me to see what I've done wrong and what I need to do to correct this.

Site has today gone over to SSL and I thought everything was ok and appears to be on face value but I just noticed the following with incoming links.
For example if you search on google, for the various subdomains Im getting different results than previously.

Google says www.kitz.co.uk yet now you end up at https://kitz.co.uk    (no www)
Google says forum.kitz.co.uk which directs to https://kitz.co.uk/forum  (want the subdomain not a folder)
Google says wiki.kitz.co.uk which now directs to https://kitz.co.uk/wiki (folder rather than the subdomain - which breaks the wikimedia software)

« Last Edit: April 28, 2018, 09:17:42 AM by kitz »
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 31543
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: DNS records help please
« Reply #1 on: April 27, 2018, 10:30:25 PM »

or is this something to do with LetsEncrypt?
Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

g3uiss

  • Reg Member
  • ***
  • Posts: 257
  • You never too old to learn
    • Midas Solutions
Re: DNS records help please
« Reply #2 on: April 27, 2018, 10:35:19 PM »

A couple of potential issues

1) google hasn’t complicated its crawl
2) if you have a site map that has been notified to google via analytics or webmaster tools has this been updated with the new it’ll ( adding the “s” )

The fact it’s resolveable in a browser suggests it’s DNS etc is fine. Google has got its links wrong. Best solution is pershaps a new site map uploaded to Google

Tony
Logged
Zen FTTC and Plusnet ADSL HG612 modems with Vigor 2925

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 31543
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: DNS records help please
« Reply #3 on: April 27, 2018, 11:09:57 PM »

Sorry I perhaps didn't explain too well the problem.

Google has all the following results cached in its search results for the site which are correct and how I want them to be

www.kitz.co.uk
forum.kitz.co.uk
wiki.kitz.co.uk

Yet if you click on any of those links you end up at the following respectively - which isnt what I want.

https://kitz.co.uk/
https://kitz.co.uk/forum
https://kitz.co.uk/wiki

So it appears to be ignoring the subdomains and I think perhaps its the redirect thats wrong.  This is part of my htaccess added by my hosts

Code: [Select]
RewriteEngine On
RewriteCond %{HTTPS} !on
RewriteRule ^(.*)$ https://kitz.co.uk/$1 [R=301,L]

#RewriteCond %{HTTP_HOST} ^kitz.co.uk
#RewriteCond %{REQUEST_URI} !^/[0-9]+\..+\.cpaneldcv$
#RewriteCond %{REQUEST_URI} !^/\.well-known/acme-challenge/[0-9a-zA-Z_-]+$
#RewriteCond %{REQUEST_URI}

Perhaps something else needs adding there, or could it be..and something which I'm beginning to suspect may not be helping, I think they may have requested just one cert for everything on the kitzcouk account.

The server was orig config in my name using WHM but is further split into 2 cpanel accounts
The first one has a couple of parked domains which arent really used for anything other than my personal mail.
The second one is where all the traffic is and anything related to the broadband site.

If I check the certificate it says

Code: [Select]
Common name: kitz.co.uk
SANs: broadbandinfo.kitz.co.uk, broadbandinfo.net, cpanel.kitz.co.uk, forum.kitz.co.uk, kitz.co.uk, mail.broadbandinfo.net, mail.kitz.co.uk, webdisk.broadbandinfo.net, webdisk.kitz.co.uk, webmail.kitz.co.uk, wiki.kitz.co.uk, www.broadbandinfo.kitz.co.uk, www.broadbandinfo.net, www.forum.kitz.co.uk, www.kitz.co.uk, www.wiki.kitz.co.uk
Valid from April 27, 2018 to July 26, 2018

which is basically everything on the 2nd account.

Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

Weaver

  • Addicted Kitizen
  • *****
  • Posts: 6053
  • Retd sw dev; A&A; 4 × 7km ADSL2; IPv6; Firebrick
Re: DNS records help please
« Reply #4 on: April 28, 2018, 02:01:28 AM »

I don't understand the first rewrite rule in your .htaccess. $1 is going to be the whole match string, so if you pass it "fred" it will just convert that to "https://kitz.co.uk/fred" won't it?

have I misunderstood?

If something like fred is being passed to it, then what about
"^([^.]+)$" to "https://$1.kitz.co.uk"

or even

"^(http:\/\/)?([^.]+)(.kitz.co.uk)?$" to "https://$2.kitz.co.uk"

Do forgive me as I suspect I haven't understood at all.
Logged

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5416
Re: DNS records help please
« Reply #5 on: April 28, 2018, 08:20:56 AM »

(edited post to reduce confusion)

Ok here is what should be happening with the rewrite. I hate regex tho makes my head explode when looking at it.

RewriteCond %{HTTPS} !on

matches anything not https, so matches http

RewriteRule ^(.*)$ https://kitz.co.uk/$1 [R=301,L]

Now this is matching anything with a . in the name so a hostname and forcing it to the static https://kitz.co.uk/ address.
However it is not fully static it has the $1 variable which matches the first part of the name, the subdomain basically as weaver said.

This is why I initially thought was ok as I am originally accessing on https so I am not hitting the rewrite condition.
So basically when google crawls the https urls it will not be affected by the rewrite rule, but it will be when crawling on http.

Try this

Code: [Select]
  RewriteCond %{HTTPS} off
  RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

HSTS may also be a consideration for enforcing https, but make sure you understand the downsides of it.
https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security
https://www.steveworkman.com/performance/2016/hsts-a-no-nonsense-guide/

I also just noticed you did a 301 not a 302 which might bite you if google bot caches it. :(
https://blog.mythic-beasts.com/2015/06/15/the-hazards-of-301-permanent-redirects/
« Last Edit: April 28, 2018, 08:43:30 AM by Chrysalis »
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 31543
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: DNS records help please
« Reply #6 on: April 28, 2018, 09:17:06 AM »

Quote
I don't understand the first rewrite rule in your .htaccess. $1 is going to be the whole match string, so if you pass it "fred" it will just convert that to "https://kitz.co.uk/fred" won't it?

I think that may have been inserted by cpanel then because I didn't put it there.   I don't understand the rewrite rules so havent touched anything, despite me spending ages last night trying to find out what to do for the subdomains.

Quote
can you provide a link kitz to the google page that has the search results, either provide the search terms or a direct link to the page.

Its any & all google search results.   https://www.google.com/search?q=kitz

hover over the results and you will see the main site takes you to www.kitz.co.uk, hover over kitz forum and it should take you to forum.kitz.co.uk etc

For the wiki look at this search result  https://www.google.com/search?q=kitz+wiki+bcm  which gives the url http://wiki.kitz.co.uk/index.php/BCM_routers yet look what happens when you click on it you end up at https://kitz.co.uk/wiki/index.php/BCM_routers which is broken because the wiki is configured to be on a subdomain.

----------------------------------------

Perhaps this explains its better,

1) If I type http://www.kitz.co.uk in my browser then you end up at https://kitz.co.uk/  - I'm ok with that
2) If I type http://forum.kitz.co.uk in my browser then you end up at https://kitz.co.uk/forum/ -  I don't want this

What I think I perhaps need then is some sort of 301 rewrite rule to ensure that any old links from the past redirects to the correct location?

I need something which does a 301 redirect/rewrite of any old forum urls that will be scattered around the internet everywhere to end up https://forum.kitz.co.uk  and similar with the wiki.  This is what I don't know how to do and despite searching I can't find anything concrete on how to handle subdomains. 

Do I need to put something in a htaccess file in root of the forum directory and wiki directory?



Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

kitz

  • Administrator
  • Senior Kitizen
  • *
  • Posts: 31543
  • Trinity: Most guys do.
    • http://www.kitz.co.uk
Re: DNS records or htaccess help please
« Reply #7 on: April 28, 2018, 09:25:28 AM »

PS   I just noticed you edited your reply whilst I was typing mine.   I definitely want a 301 as they are permanent not temporary moves

Quote
When permanently moving a web site, or a web page, best practice is to use a 301 redirect. 302s in this situation seem incorrect. By saying “temporary move” a 302 tells search engines to keep the old domain or page indexed, but it would be desirable for them to index the new location. In the past people have used 302 redirects in an effort to circumvent the Google aging delay. This workaround might have worked at some point, but it is not a current best practice.

Whether Google actually handles 302s properly is an open question. If a 302 is used instead of a 301, search engines might continue to index the old URL, and disregard the new one as a duplicate. Link popularity might be divided between the two urls, hurting search rankings.

Logged
Please do not PM me with queries for broadband help as I may not be able to respond.
-----
How to get your router line stats :: ADSL Exchange Checker

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5416
Re: DNS records or htaccess help please
« Reply #8 on: April 28, 2018, 09:28:23 AM »

yes but 301 only after its tested its the behaviour you want. As it locks it in place.
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5416
Re: DNS records or htaccess help please
« Reply #9 on: April 28, 2018, 09:29:31 AM »

so to do as I said

remove these lines

RewriteCond %{HTTPS} !on
RewriteRule ^(.*)$ https://kitz.co.uk/$1 [R=301,L]

add these lines

RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

All it should do is replace http with https and thats it, no changing of the url. You only actually need to swap one line, but just do both anyway. 
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

jelv

  • Helpful
  • Kitizen
  • *
  • Posts: 1014
Re: DNS records or htaccess help please
« Reply #10 on: April 28, 2018, 09:43:40 AM »

It's not just Google that is wrong. I have http://forum.kitz.co.uk/index.php?action=unread in my browser bookmarks and I end up at https://kitz.co.uk/forum/index.php?action=unread

I didn't notice at first because the forum software corrects it when you click the first link.
Logged
Line rental: Pulse8, Broadband: AAISP Home::1 FTTC 80/20, Mobile: id Mobile

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5416
Re: DNS records or htaccess help please
« Reply #11 on: April 28, 2018, 09:45:30 AM »

kitz read on the link I gave you below this line

"Expiring permanent redirects"

That seems a sane solution to the search engine problem.

so e.g.

Code: [Select]
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,E=limitcache:1]
Header always set Cache-Control "max-age=3600" env=limitcache
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

Chrysalis

  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5416
Re: DNS records or htaccess help please
« Reply #12 on: April 28, 2018, 09:47:32 AM »

It's not just Google that is wrong. I have http://forum.kitz.co.uk/index.php?action=unread in my browser bookmarks and I end up at https://kitz.co.uk/forum/index.php?action=unread

I didn't notice at first because the forum software corrects it when you click the first link.

yeah this will be ok when the htaccess is changed.

If the original link is https there is no redirect at all, so its only a problem when the original link is http.
Logged
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE

jelv

  • Helpful
  • Kitizen
  • *
  • Posts: 1014
Re: DNS records or htaccess help please
« Reply #13 on: April 28, 2018, 09:48:23 AM »

I Googled "force http to https htaccess" and found

https://www.namecheap.com/support/knowledgebase/article.aspx/9770/38/how-to-force-https-using-htaccess-file-in-cpanel

Does that help?

There's loads of other results on the same search.
Logged
Line rental: Pulse8, Broadband: AAISP Home::1 FTTC 80/20, Mobile: id Mobile

jelv

  • Helpful
  • Kitizen
  • *
  • Posts: 1014
Re: DNS records or htaccess help please
« Reply #14 on: April 28, 2018, 09:50:05 AM »

yeah this will be ok when the htaccess is changed.

I know - I was reading the part of the beginning of the topic where Google crawl etc was mentioned.
Logged
Line rental: Pulse8, Broadband: AAISP Home::1 FTTC 80/20, Mobile: id Mobile
Pages: [1] 2 3 4
 

anything