--- Log opened Fri Feb 08 00:00:04 2019
04:54 < aestetix> anyone here good with varnish?
06:02 <@Dolemite> mr0ning, be0tches and h0ez!
06:02 <@Dolemite> aestetix: Yes.   They key is to only apply a thin layer with each coat, and let that coat dry completely before you apply the next one. ;)
06:09 <@Shadow404> aestetix: sniffing?
06:09 <@Shadow404> alo, wipe off any remnants of the previous coat with a low lint cloth
06:09 <@Shadow404> *also
06:11 < aestetix> I meant the software
06:11 < aestetix> but now I have resolved my issue
06:25 < fie> "The project was initiated by the online branch of the Norwegian tabloid newspaper Verdens Gang."
06:25 < fie> well I'll be...
07:26 -!- opticron [~opticron@gateway.oarcorp.com] has joined #se2600
07:26 -!- mode/#se2600 [+o opticron] by ChanServ
08:26 -!- K`Tetch_ [~no@47.39.211.239] has joined #se2600
08:30 -!- K`Tetch [~no@unaffiliated/ktetch] has quit [Ping timeout: 246 seconds]
11:17 <@dasunt> I prefer working with anal-retentive people when it comes to server management.
11:18 <@dasunt> This whole "lets wing it" approach we apparently have hear is pissing me off for the amount of work it causes.
11:19 -!- crashcartpro [uid29931@gateway/web/irccloud.com/x-dgkwjdmosrxymozg] has joined #se2600
11:19 < xray> details matter
11:20 <@dasunt> Whoever says consistancy is the sign of a small mind never did administration.
11:20 <@Mirage> dasunt: "lets wing it" works best when the person performing the action has a clue.
11:21 <@dasunt> "Lets wing it" does great when the person winging it above me in the office food chain, and I'm the one that gets to fix all of the issues.
11:21 <@dasunt> They get everything done quickly, and I'm the one stuck with problems that takes hours to solve or mitigate.
11:22 <@dasunt> It's like working in a small programming environment where a programmer adds a new feature, does one test, and decides it works.
11:22 <@Mirage> dasunt: you just need a bunch of India based drones.. they're execellent for "just follow what's documented and freak out if anything isn't exactly as documented"
11:23 <@dasunt> Mirage: Sadly, that doesn't sound like it's a bad thing.
11:24 <@dasunt> But see, I also have a bit of that in the "wing it" boss.  Since I've been harping on following the procedure, she just does it and doesn't care if huge error messages flash.
11:24 <@dasunt> (Then she skips the testing part of the instructions, since that would reveal mistakes)
11:25 <@Mirage> It kills me when our India L2's raise an issue because they logged into a server and instead of a bash prompt like "[root@zqkpcloudxm9013 report-scripts]#", they see one that's been changed to either just "bash #" or "root@zqkpcloudxm9013 [/usr/local/billingReports/report-scripts] #"
11:26 <@dasunt> All I want is that for tasks we have to do multiple times, we do them the same way, and if errors happen, we investigate them instead of ignoring them.
11:27 <@Mirage> For things like server builds, etc we have a configuration validation script that they are required to run and attach the output of to the build ticket when they close it.
11:30 <@Mirage> For LVM I had to write a shell script with tons of logic and error handling that wound up being over 700 lines long..and fully colorized because they kept fucking up creating/extending/formatting.  Blew my mind how they just couldn't get how LVM works since it's been around for a long time and took me all of about 10 minutes to learn
11:32 <@Mirage> oops.. s/700/400/
11:33 <@Mirage> https://pastebin.com/e3Exck4Y
11:33 < PigBot> [Bash] #!/bin/bash if [ "`whoami`" != "root" ]; then echo "This script must - Pastebin.com (at pastebin.com) http://tinyurl.com/yb2p7xf5
11:41 <@Evilpig> Mirage: wouldn't it save you a little time to pass -r to your lvextend so it resizes at the same time?
11:41 <@Evilpig> instead of coming back to call resize2fs
11:42 <@Evilpig> it'd also let you add in xfs as a format option instead of just using ext as well
11:46 <@Mirage> Evilpig: I wrote that in 2014
11:48 <@Mirage> Noone has said anything about needing to add xfs, so I assume the customer that was primarily written to be used for isn't using it yet.  Wouldn't suprise me...they still had RHEL3 hosts in production 2 years ago
11:48 <@Evilpig> that'd do it
11:49 <@Evilpig> the day I learned of that -r flag it was like voodoo
11:49 <@Evilpig> happy delightful, take my soul, voodoo
12:01  * dasunt ponders setting himself on fire.
12:11 <@Corydon76> dasunt: do you raise awareness with the person who decided to 'just wing it' every time you discover something which is broken as a result?
12:11 <@Corydon76> If the guy (isn't it always a guy?) is unaware, it may be the reason he keeps throwing caution to the wind.
12:20 < aestetix> yes, because everyone knows women never make mistakes
12:31 <@dasunt> This is more of an information flow problem.  Specifically one of killing the messenger.
12:59 <@Mirage> There's always the "make the person who broke it fix it" option
13:07 <@_NSAKEY> Mirage: I think the problem here is that dasunt's idiot manager is also the chaos monkey who keeps breaking things.
13:07 <@_NSAKEY> lol, let's all be glad we don't work for wells fargo https://www.reddit.com/r/sysadmin/comments/ao4g2y/wells_fargo_is_down_declining_transactions_and_no/
13:08 < PigBot> Wells Fargo is down. Declining transactions and no access to the web portal. : sysadmin (at www.reddit.com) http://tinyurl.com/yc2nevsj
13:17 <@dasunt> Wait, fire suppression went off and now they are down?
13:18 <@dasunt> Shouldn't WF have a failover system?  I thought that was the norm for big banks - main datacenter location, secondary datacenter location that's ready to go at a moment's notice, etc.
13:18 < K`Tetch_> i'll remind you, we're talkign wells fargo
13:19 < K`Tetch_> besides, it'd mean th skimming and false accounts would have to be deliberately mirroed ont h second site, which shows premeditation
13:21 <@dasunt> TN will be paying $15.2M in incentives for 5,000 Amazon jobs?
13:21 <@dasunt> So $3k per job?
13:25 <@Evilpig> I read about that wells fargo thing yesterday
13:25 <@Evilpig> failure to do dr tests and then have your primary dc shutdown due to the fire supression system getting tripped
14:32 <@dasunt> Yep, seems sloppy.
14:55 -!- opticron [~opticron@gateway.oarcorp.com] has quit [Quit: leaving]
15:18 <@dasunt> Speaking of which, I know there's something like 250,000 visitors, but the reservation system for the BWCA is still down and is planned to be down for a month.
15:19 <@dasunt> ISTM that it should not be that hard.
15:19 <@dasunt> Old online system worked just fine.
15:27 < aestetix> https://www.youtube.com/watch?v=02ux1dKNPXo
15:27 < PigBot> Joe Rogan Experience #1242 - Tim Pool (at www.youtube.com) http://tinyurl.com/y9v6qyvp
15:35 -!- opticron [~opticron@shop.MAKERSLOCAL.org] has joined #se2600
15:35 -!- mode/#se2600 [+o opticron] by ChanServ
15:49 <@_NSAKEY> Evilpig: I'm pretty sure the Fed will skullfuck them over the DR stuff.
15:50 <@_NSAKEY> If they're found to not have done the quarterly DR tests or to have lied about them, that's more fines.
15:52 <@_NSAKEY> I just double-checked, and Wells Fargo is a G-SIB. I can't imagine them not being forced to do quarterly DR testing, with at least one test a year featuring a cameo appearance by someone from the Federal Reserve.
15:52 <@_NSAKEY> Then again, K`Tetch_ is likely right. We're talking about Wells Fargo.
15:56 <@dasunt> It just seems really bad.
15:58 <@dasunt> Correct me if I'm wrong, but isn't the normal circumstance for businesses where any downtime can be mundo expensive is to have a secondary location, always hot, being kept in sync with the primary, and ready to go at a moment's notice?
15:59 <@dasunt> I remember seeing a presentation for a CC company, and they had their primary site, their hot secondary site (different part of the country, IIRC), then offline tertiary and quaternary sites.
16:11 -!- K`Tetch [~no@unaffiliated/ktetch] has joined #se2600
16:12 -!- K`Tetch_ [~no@47.39.211.239] has quit [Ping timeout: 246 seconds]
16:19 -!- K`Tetch_ [~no@47.39.211.239] has joined #se2600
16:22 -!- K`Tetch [~no@unaffiliated/ktetch] has quit [Ping timeout: 250 seconds]
16:25 -!- K`Tetch [~no@unaffiliated/ktetch] has joined #se2600
16:28 -!- K`Tetch_ [~no@47.39.211.239] has quit [Ping timeout: 245 seconds]
16:32 <@_NSAKEY> dasunt: That's when you're dealing with gold-plated solutions, but Wells Fargo likely optimized for defrauding their customers at web scale over implementing good DR
16:33 <@_NSAKEY> Big banks are supposed to test DR failover regularly, but the testing is of course under artificial constraints (There's time to prepare for it).
16:33 <@_NSAKEY> And, good luck convincing a big bank to implement something like chaos monkey.
16:34 <@_NSAKEY> Maybe for some sort of new infrastructure, assuming the C-suite is feeling particularly brave or in the mood to grab headlines for being bold, but not for the legacy stuff.
16:47 -!- K`Tetch_ [~no@47.39.211.239] has joined #se2600
16:50 -!- K`Tetch [~no@unaffiliated/ktetch] has quit [Ping timeout: 272 seconds]
18:06 -!- K`Tetch [~no@unaffiliated/ktetch] has joined #se2600
18:10 -!- K`Tetch_ [~no@47.39.211.239] has quit [Ping timeout: 246 seconds]
18:24 -!- K`Tetch_ [~no@47.39.211.239] has joined #se2600
18:27 -!- K`Tetch [~no@unaffiliated/ktetch] has quit [Ping timeout: 240 seconds]
19:25 -!- K`Tetch [~no@unaffiliated/ktetch] has joined #se2600
19:29 -!- opticron [~opticron@shop.MAKERSLOCAL.org] has quit [Quit: leaving]
19:29 -!- K`Tetch_ [~no@47.39.211.239] has quit [Ping timeout: 268 seconds]
20:00 -!- opticron [~opticron@75.76.209.103] has joined #se2600
20:00 -!- mode/#se2600 [+o opticron] by ChanServ
20:47 -!- ware_ [ware@phneak.net] has quit [Remote host closed the connection]
22:11 -!- crashcartpro [uid29931@gateway/web/irccloud.com/x-dgkwjdmosrxymozg] has quit [Quit: Connection closed for inactivity]
22:22 -!- K`Tetch_ [~no@47.39.211.239] has joined #se2600
22:22 -!- K`Tetch_ [~no@47.39.211.239] has quit [Changing host]
22:22 -!- K`Tetch_ [~no@unaffiliated/ktetch] has joined #se2600
22:26 -!- K`Tetch [~no@unaffiliated/ktetch] has quit [Ping timeout: 268 seconds]
22:30 -!- K`Tetch [~no@47.39.211.239] has joined #se2600
22:30 -!- K`Tetch [~no@47.39.211.239] has quit [Changing host]
22:30 -!- K`Tetch [~no@unaffiliated/ktetch] has joined #se2600
22:33 -!- K`Tetch_ [~no@unaffiliated/ktetch] has quit [Ping timeout: 272 seconds]
23:42 -!- opticron [~opticron@75.76.209.103] has quit [Quit: leaving]
--- Log closed Sat Feb 09 00:00:05 2019