Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[kitz]

Maybe only a new post is affected, not replies.

It shouldnt only be new posts. 
I created a brand new test post a couple of nights ago in the FTTP section that includes inverted commas
https://forum.kitz.co.uk/index.php/topic,28096.0.html

Both of the following are standard  chars should work fine.  In fact any of the standard ASCII characters work ok

Code: [Select]

DEC OCT HEX BIN Symbol HTML Number HTML Name Description
34 042 22 00100010 " " " Double quotes (or speech marks)
39 047 27 00100111 ' ' ' Single quote
96 140 60 01100000 ` ` ` Grave accent

Most UK keyboards just have a single & double inverted comma (  ' and " ) which should be OK.

------------------------------------
The problem is non-standard ASCII chars. 
During previous testing the only character on a UK keyboard that has a problem is the UK currency pound sign.   The pound symbol isnt a standard ASCII key.  There are a few other non standard ASCII keys that could case problems such as the Euro symbol, the copyright symbol and the registered trademark.  Although not on the keyboard they can be inserted using the designated number and the ALT key

For example

Code: [Select]

* Alt+0169 Euro sign
* Alt+0174 Registered
* Alt+0169 Copyright

-------------------------------
 From the info in the error log yesterday it looks like the error is being caused by the single right hand quote mark. 

This character is not on the keyboard nor is it a standard ASCII no.  It's generated by certain software - usually Windows- such as Word or perhaps a html editor like Dreamweaver which uses the html ref no. If you copy and paste a character that has at some point in the past been edited in Windows software to use the html ref then it will throw the error.

We - or rather Meritas - has previously seen a problem with the left and right double quotes. Because all your error messages all say "Incorrect string value: '\x92" then I think its a fair assumption to say that it is char hex 92 aka single right single quote.   Replacing the windows generated guote marks with the ASCII symbols [ ' ] should fix it.


Windows left and right quotations marks

Code: [Select]

DEC OCT HEX BIN Symbol HTML Number HTML Name Description
145 221 91 10010001 * ‘ ‘ Left single quotation mark
146 222 92 10010010 * ’ ’ Right single quotation mark
147 223 93 10010011 * “ “ Left double quotation mark
148 224 94 10010100 * ” ” Right double quotation mark


[roseway]

My keyboard has got a grave accent on the key to the left of the number 1. Here it is: `

[Alex Atkin UK]

I assume you've seen this? https://nicj.net/mysql-converting-an-incorrect-latin1-column-to-utf8/

[aesmith]

I've just created a post in the FTTP rollout forum. Pasting in my text, sentence at a time, I can see at what point it errors.

The digital engagement team have access to the government's local authority portal so I can confirm that superfast fibre broadband will be installed to your property in 2025.

[kitz]

My keyboard has got a grave accent on the key to the left of the number 1. Here it is: `

Thank you, I think that although the grave key ` might look similar to the left single quotation mark, it should be fine as the grave is a standard 127 ASCII char . Or at least it seems to work ok in a windows environment and I'd grouped it above in with the standard single and double quotes as one of the keys that worked fine for me.

I'm not sure about some things when it comes to linux as there may be chars it treats differently?

I think the chars that Meritez had problems with were symbols from elsewhere that a Windows software app or dreamweaver erc had used the left and right double quotation marks.... I think this is similar only with the single left and right.

---
Was going to add more but Ive just hit a blank wall whilst doing this post and another.

[kitz]

I assume you've seen this? https://nicj.net/mysql-converting-an-incorrect-latin1-column-to-utf8/

Thank you I have just had a read of that which is a good guide..
Unfortunately, after I reported that the migrated data had come across as  latin and they attempted to fix it by updating the tables to utf8then we are about here:-

If you simply force the column to UTF-8 without the BINARY conversion, MySQL does a data-changing conversion of your "latin1" characters into "UTF-8" and you end up with improperly converted data. Converting the column to BINARY first forces MySQL to not realize the data was in UTF-8 in the first place.

To make things worse when I attempted to roll back to my backups SMF things got a bit nasty.  The front page of the forum totally disappeared and there was a problem if someone had previously used one of the non standard chars.   As far as the server was concerned there became some chars that no longer existed.

Sorry I cant type any more. I was also making another post but hit the wall when trying to type an iso no and now I cant think to complete this one.  Im so frustrated with this damn disease at how slow it makes me and how a simple thing can just shut my brain down.

But aside from the conversion going wrong and the attempted reupdate to utf8 again, theres a conflict with the server software that has problems with utf8 now.  Theres also a problem with my brain which needs rewiring

[roseway]

I'm not sure about some things when it comes to linux as there may be chars it treats differently?

There's no simple answer to that, because Linux comes in many different flavours, and is also highly configurable by the user or system administrator. For English speaking regions ISO8859-1 is common, but a lot of modern systems also support UTF-8. As far as this forum is concerned it's probably true to say that if Windows can handle it then so can Linux.

[kitz]

I've just created a post in the FTTP rollout forum. Pasting in my text, sentence at a time, I can see at what point it errors.

From the logs I can see you were getting this same error x5

Code: [Select]

Incorrect string value: '\x92s loc...' for column 'body' at row 1

x92 appears to mean hex 92, this converts to the right single quote mark.

That would indeed appear to fit with what you say you managed to post.  ie someone stated something but you couldnt post what they stated.    Possibly  because it was wrapped with the single LH and RH quotes.   Just change those quote marks to standard quote marks ' or " and it should be fine.   

[Alex Atkin UK]

Thank you I have just had a read of that which is a good guide..

Its why I thought I should post it, as it seemed to explain where things went wrong with your last attempt.  Its such a PITA.

But aside from the conversion going wrong and the attempted reupdate to utf8 again, theres a conflict with the server software that has problems with utf8 now.  Theres also a problem with my brain which needs rewiring

My brain is pretty useless for much else than computer stuff, and not even that on a bad day, its why I've never taken on something like a forum as I know it would be a tough thing to keep on top of.  My mum is losing the ability to find the words to describe things, plus falling asleep all the time.  So I can certainly relate on some level.

[kitz]

My mother has Alzheimer's with poor memory especially recent events. The neuro diseases such as MS are a bit different in that I know what I want to say and the stuff is there, but its like a wall comes up and it becomes arduous to find the words to communicate those words that you want to say.  I used to say I feel like it's robbed my vocabulary. I feel its worse than Alzheimer's in that my mother doesn't know she has a problem and she thinks shes fine and in the best of health.  If she forgotten something that caused a problem, then a few mins later she's forgotten there was a problem, so she carries on with life happy enough.

The neuro diseases where you get a wall or fog is exceedingly frustrating because you are fully aware that you are struggling to communicate.  The words are flying round your head and you can almost visualise them but its like they are stuck unable to say or write the word.  It isnt necessarily long or complicated thought, it can sometimes be a common word. 
I get so frustrated. I have to concentrate to speak, so I may not be able to walk and talk at the same time. Quite often the words will jumble... and I might know that they are jumbled but struggle to put them in the correct order. *
An auto immune disease where your body attacks the central nervous system.. Basically it is a transmission fault with the neural pathways.  Some of the tests they do are able to measure the time taken between points A & B and the amount of damage is shown as latency.  I can get distracted easily and then thinking about too many things its like congestion where everything can grinds to a halt.

Like now, I know that my first paragraph is jumbled and needs rephrasing to better put my point across and smoother reading.  Some days I may type it in notepad and come back later, but its very late and its taken me longer than it should have and I want to get to bed.

[Alex Atkin UK]

The neuro diseases such as MS are a bit different in that I know what I want to say and the stuff is there, but its like a wall comes up and it becomes arduous to find the words to communicate those words that you want to say.

The neuro diseases where you get a wall or fog is exceedingly frustrating because you are fully aware that you are struggling to communicate.  The words are flying round your head and you can almost visualise them but its like they are stuck unable to say or write the word.  It isnt necessarily long or complicated thought, it can sometimes be a common word. 

That's pretty much how my mum is describing it, that's its there, she can see it in her head but can't get it out.  Also when she is trying to do anything, any distraction and she loses her train of thought, which often ends up in her yelling at me to shut up.  Its frustrating on both sides as I instinctively want to try to help, but if I do it makes it worse.

A few weeks ago she completely forgot how to get into her Internet banking and we've still not managed to sort that out.  I've taken over the electricity/gas bill, but anything else we have no idea if its been paid or not at this point as she has no clue how much money was in her current account.  I can't physically carry a lot of shopping, or know necessarily what she needs so its extremely problematic - we've been mostly living of the contents of the cupboard and freezer.

What triggered this was me moving to Universal Credit causing the council to want her to write in to clarify details for them to continue paying Housing Benefit.  So I was hardly in a good place to start paying all the bills.  Fortunately I spent the last year paying off most of my debts and cut all unnecessary spending for the past month to build up a safety net.