Caravan and Motorhome Club's website down! (1 Viewer)

[starquake]

If you have database on that scale, then you run multiple replication nodes live. You run on a SAN and have multiple hot nodes ready to go.
CAMC are not at that scale nor have that kind of budget,

Or you use cloud technology that allows that at more modest budgets -> standard service offering on most of them. But thats not our call! I agree it can be very expensive in some cases -> as SAN replication of the type you mention is very very expensive.

I ended up managing Database professionals during my last years in computer employment. The one thing I learnt is that a good DBA costs and is worth their weight in gold. There are a lot who claim to be very capable but the real experts need to be paid well, not something every business is willing to do.

And totally agree with this -> DBA's are the key to good business. And many organisations (inclding one I work for now) seem to think the profession isn't needed which is quite quite nuts. Has lead on a current project to them having numerous security weaknesses that I have to write up this week. I have the most respect for DBA's as in one case on a project ~ 8 years ago, they stopped a hardware upgrade project costing near half a million for that app instance alone (they had multiple instances of same system, so total cost would have been millions for ALL instances) to be scrapped as the reason for long transaction times was not the database hardware, it was locks in the 3 page SQL transaction across the database causing a commit delay. They fixed that, suddenly the app performed without any hardware upgrade with transaction times in milliseconds instead of seconds. It staggered the then management that the vendor they selected for their systems didn't have a DBA able to spot that contention in their (admittedly complex) code. DBA's save millions in some organisations for sure!

 

[dna]

They won't hold card details. The PCI/DSS requirements have ramped up to stupid levels in recent years. Very few companies will even risk holding client card details unless they are suicidal and don't want business insurance.

I don't know the company they use, but you are correct it will be someone like WorldPay that does it for them

BUT, This is not what is causing the issues. If it was a payment issue you would still be able to book just not pay.


They do not need IT expertise. They need campsite expertise. These days no company of their size or smaller should be doing their own IT. The security and stability requirements just too high for such a non core activity.
I suspect they have an "IT" contract and something has gone wrong and they are being pushed from pillar to post. I suspect the new site was developed and hosted under a single contract and they are unwilling at this stage to throw the contractor under the bus until they are sorted. But I suspect at contract renewal time the current company will not be in the running.

Gromett I understand what you mean by IT should be a purchased service. However, the Club should have enough expertise to comprehend the service they are offered and to set the appropriate requirements. It sounds expensive but some specialist consultancy on setting the requirements and being able to vet any proposals could have been a godsend when choosing the solution and understanding what they have actually bought. It would be good if once the systems are back, that the club engage an independent IT literate person / company to run a detailed autopsy on what happened and get the lessons learnt. (dont let the current IT supplier lead on that!)

As another ex IT worker I spent some time in Service Management for big banks / insurance companies. It was always hard to tie business project people down to decisions on downtime / recovery / data loss and get them to appreciate the cost of those things balanced against the cost of the precautions. Some business applications warranted 99.999% availability 365 days a year but as you would know the solutions cost a fortune to deliver less than 6 minutes a year off line.

The CAMC won't need anything like that availability, maybe they should at least inform the membership that the target recovery time for the booking system is actually x working days.

 

[FrankNicklin]

I would say the user interface part of their IT is only average but can see its a matter of opinion. The back end seems fine.

Regardless, their communication is often poor. It would not be difficult to put an explanation on the holding page we currently see with either a warning customer data may be compromised, or reassurance that it is not. Equally they could get a message to this site and the Motorhome magazine sites with similar information. That would either allow people to stop worrying or change passwords elsewhere which they may have unwisely replicated. Let’s face it most people do this.

Finally some sort of holding message is in their own best interests, it would help to damp down some of the wilder speculation.

If they make statements about the possibility of data being compromised they could leave themselves open to an investigation by the ICO. If data has been compromised they have to refer themselves to the ICO. The financial punishment should this occur can be based on a percentage of their earning and be hefty.

 

[FrankNicklin]

They won't hold card details. The PCI/DSS requirements have ramped up to stupid levels in recent years. Very few companies will even risk holding client card details unless they are suicidal and don't want business insurance.

I don't know the company they use, but you are correct it will be someone like WorldPay that does it for them

BUT, This is not what is causing the issues. If it was a payment issue you would still be able to book just not pay.


They do not need IT expertise. They need campsite expertise. These days no company of their size or smaller should be doing their own IT. The security and stability requirements just too high for such a non core activity.
I suspect they have an "IT" contract and something has gone wrong and they are being pushed from pillar to post. I suspect the new site was developed and hosted under a single contract and they are unwilling at this stage to throw the contractor under the bus until they are sorted. But I suspect at contract renewal time the current company will not be in the running.

A lot of companies believe themselves to be PCI/DSS compliant when they are not. It’s business suicide if they do hold this data.

 

[Otter Spotter]

If they make statements about the possibility of data being compromised they could leave themselves open to an investigation by the ICO. If data has been compromised they have to refer themselves to the ICO. The financial punishment should this occur can be based on a percentage of their earning and be hefty.

Watch the news some time in the future, CAMC being investigated by ICO

^{_{Subscribers  do not see these advertisements}}

 

[Coolcats]

Lots of business phones are voice over internet

Everything will be voip shortly no analogue or digital. All BT exchanges will be switched off.

 

[Coolcats]

The level of CAMC IT expertise is seriously lacking, demonstrated after the recent new website launch. I hoped it’s not been hacked, but wouldn’t be surprised. In which case if there’s been a data breach they need to make an official statement and report themselves to the relevant authorities ASAP.

this always makes me chuckle it’s like reporting your self to the headmaster for a caning…the headmaster knows you have done wrong it’s just you get 6 of the best instead of 12 if you get to the office before your dragged there

 

[Wee Bold Davy]

When I renewed my membership last month, I insisted on having a membership card as I refuse to put my whole life in a phone. I've not had the club magazine in the post for around 4 months, I wasn't given the choice of reading it online, they just stopped sending it. They must be saving a fortune by not printing and mailing them, yet the membership cost more and the cost of using the club sites have rocketed in the past two years.
I don't suppose with the cost of electricity falling that it will make a bit of difference to what we're paying.

 

[CAB96]

Just phoned Commons Wood site and was told they could not take bookings until the system was back up. I was hoping to use my voucher but have had to book elsewhere.

Calor had a system update last year, the web site was supposed to be down for one weekend but didn’t reappear for months. If the C&MC site is down that long they are going to lose an awful lot of business but it will make cutting the grass easier. They might even have to authorise the purchase of paper diaries and planners for all the sites though I imagine they would have to call a board meeting first.

Is this the free night voucher?

You had to have booked before 31st December for it to be valid.

 

[FrankNicklin]

Everything will be voip shortly no analogue or digital. All BT exchanges will be switched off.

Yep and the rollout has been a disaster. People losing cherished telephone numbers. People not realising that with a power cut you no longer have a telephone service because your internet will be offline so you have to invest in a UPS, absolutely critical for medical equipment monitored and used at home. In the name of progress you go from a little white box on the wall to an ONT, then a router, then a telephone adapter if the router isn't compatible with your analog phone, then a UPS if you want to keep it all running during these threats of increased power outages. Those that have never needed an internet service will have no choice just to have a house telephone. Not everyone can operate a mobile.

^{_{Subscribers  do not see these advertisements}}

 

[FrankNicklin]

Well according to their privacy statement I got from their site via another route (limited access to services so don’t ask for the link) they do store bank account details but do not hold card payment details. They also state they are PCI DSS compliant.

 

[Coolcats]

Yep and the rollout has been a disaster. People losing cherished telephone numbers. People not realising that with a power cut you no longer have a telephone service because your internet will be offline so you have to invest in a UPS, absolutely critical for medical equipment monitored and used at home. In the name of progress you go from a little white box on the wall to an ONT, then a router, then a telephone adapter if the router isn't compatible with your analog phone, then a UPS if you want to keep it all running during these threats of increased power outages. Those that have never needed an internet service will have no choice just to have a house telephone. Not everyone can operate a mobile.

Oh er not sure its such a disaster, but this is one issue of market competition in that not all providers are equal when you have a state run Telco with a universal provision stuff tends to be standard. There are those who do not want the telco's router and buy their own 'it's better' (what ever that means). If you have an old BT router they will provide you with one that has a phone port and a cable or just a cable to plug your analogue phone in to.

Phone exchanges provided power to your phone, now you have to provide your own power if there is a power cut, as I do for my Broadband and NAS and office if you need UPS APC offers a range .

Less and less people have land lines

Technology and IP networks have changed everything other countries are in front of us back in 2016 going around Iceland everywhere had IP phones all the shops and business's

Will it affect me yes, as my FAX machine won't work anymore (doesn't work over IP networks)

 

[Coolcats]

Gromett I understand what you mean by IT should be a purchased service. However, the Club should have enough expertise to comprehend the service they are offered and to set the appropriate requirements. It sounds expensive but some specialist consultancy on setting the requirements and being able to vet any proposals could have been a godsend when choosing the solution and understanding what they have actually bought. It would be good if once the systems are back, that the club engage an independent IT literate person / company to run a detailed autopsy on what happened and get the lessons learnt. (dont let the current IT supplier lead on that!)

As another ex IT worker I spent some time in Service Management for big banks / insurance companies. It was always hard to tie business project people down to decisions on downtime / recovery / data loss and get them to appreciate the cost of those things balanced against the cost of the precautions. Some business applications warranted 99.999% availability 365 days a year but as you would know the solutions cost a fortune to deliver less than 6 minutes a year off line.

The CAMC won't need anything like that availability, maybe they should at least inform the membership that the target recovery time for the booking system is actually x working days.

99.999 reliability should be pretty standard today given technology and its reliability (if set correctly)

 

[starquake]

99.999 reliability should be pretty standard today given technology and its reliability (if set correctly)

I dunno as cloud providers don't offer this level unless you architect for multiple AZ and in some cases regions for that. https://aws.amazon.com/compute/sla/. On a single "server" they only offer 3 9's.

There is a seperate SLA by service, and it does get expensive for 3 9's and even more for 4!

 

[BillandHelen]

Well, looks like the tech guys are making some progress, we just got our automated invoice email for our stay, which means that wardens must have access to the system, guess they are in soft relaunch mode. Hopefully everything goes well for them,

^{_{Subscribers  do not see these advertisements}}

 

[Gromett]

Or you use cloud technology that allows that at more modest budgets

We will have to disagree there. Cloud is never for more modest budgets. It is expensive as hell in my experience.

 

[FrankNicklin]

We use hosted dedicated servers in our line of business and that’s for fairly basic backend services and databases And not cheap. I’ve no idea how big CAMC is or their turnover, I guess I could look on companies house, but clearly they lack a decent disaster recovery process that will need some investment. As a small business it’s not too much of a problem if your website goes down, a pain but not a show stopper. CAMC have a lot of members and campsites reliant on a working backend, so they need to invest in a better disaster recovery strategy.

 

[Langtoftlad]

A previous warden told me it was ferry booking software but neither here nor there!

When Hilton International Hotels first went computerised back in the early 80's... it soon became apparent that the software had been (poorly) adapted from a car rental company's operating system.
Threw up some bizarre quirks & glitches.

 

[FrankNicklin]

When Hilton International Hotels first went computerised back in the early 80's... it soon became apparent that the software had been (poorly) adapted from a car rental company's operating system.
Threw up some bizarre quirks & glitches.

First one was when the double room you booked turned out to be the back of a Ford Cortina estate.

 

[Langtoftlad]

First one was when the double room you booked turned out to be the back of a Ford Cortina estate.

You may laugh... but...

^{_{Subscribers  do not see these advertisements}}

 

[starquake]

Major IT outage denies happy campers their caravan holidays

1 million members still searching for answers as IT issues floor primary digital services

www.theregister.com

 

[FrankNicklin]

You may laugh... but...

No it’s no laughing matter. I still see software that’s been adapted for a purpose for which it was never intended either because nothing else was available or it was cheap at the time.

 

[Coolcats]

I dunno as cloud providers don't offer this level unless you architect for multiple AZ and in some cases regions for that. https://aws.amazon.com/compute/sla/. On a single "server" they only offer 3 9's.

There is a seperate SLA by service, and it does get expensive for 3 9's and even more for 4!

Jeepers sounds like an opportunity for a innovative company

 

[Jim]

If the Register is to be believed then it looks very much then like our data has been breached.

 

[FrankNicklin]

Jeepers sounds like an opportunity for a innovative company

We use a reputable cloud backup service that if we were using AWS instead would cost us $118,000 a year. Thats an awful lot for a small company.

^{_{Subscribers  do not see these advertisements}}

 

[chesterfield hooligan]

Not to worry it will all be covered by the increased CAMC membership fee.

 

[Gromett]

If the Register is to be believed then it looks very much then like our data has been breached.

Well if they have reported themselves to the ICO then is definitely a breach. However, The register says they have reported themselves but I have not seen this anywhere else? How do they know?

 

[Jim]

Well if they have reported themselves to the ICO then is definitely a breach. However, The register says they have reported themselves but I have not seen this anywhere else? How do they know?

The last time I looked they had a couple of hundred million in their current account, so they risk a big fine if they fall foul of the ICO

 

[Gromett]

The last time I looked they had a couple of hundred million in their current account, so they risk a big fine if they fall foul of the ICO

I am curious as to how The Registers knows they have reported themselves to the ICO. If they have had a security breach then not reporting themselves would be not only criminal but criminally stupid. You can't hide this stuff and they will always find out.

 

[Gromett]

Named and Shamed? ICO now publishing names of organisations suffering data breaches

www.dacbeachcroft.com

^{_{Subscribers  do not see these advertisements}}