Hewlett-Packard 3314A function generator

[Zenith]

Thanks. I may take you up on that.

I haven't looked at the 2019 properly yet, just powered it up, looked at the output with a scope and frequency counter, and noted a few of the problems. One of the rear feet was missing, which was surprisingly annoying, so I replaced both with a pair of generic feet from something else. At least it doesn't wobble about now.

It's about sixth on the list of major items to tackle.

[synx508]

I've poked and prodded the 3314A for a while and after replacing a 74LS245 it has resumed some sort of operation. I removed the NVRAM and have a horrific setup with a 28 pin SRAM hovering above the board with lots of wires soldered onto two IC sockets that are placed where the µPD444Cs were previously. The RAM may have been innocent, I've a feeling the '245 may have been a root cause of one of the symptoms, or perhaps all of them, as it is connected to the CPU's data bus and its job is to talk and listen to other circuit blocks. Unfortunately I don't have a functioning auto-calibration system now and this seems to be down to the awful, awful PCB interconnects that HP somehow got even more wrong than the terrible ones fitted to the 3325B. It could even be that the cables are to blame for every fault.

By examining the different PCBs I've discovered that my 3314A's external serial number date in 1981 is completely unconnected to the boards within, the newest of which seems to be from 1995. This thing has had numerous board swaps and those interconnects should've been replaced ages ago. I changed the pin headers on the computer board A3 because they were looking very rough but I didn't look closely enough at the interconnects themselves at the time. These wiring hell devices are 0.1" spaced ribbon cable with solid and very breaky cores. The result of this is that I had a bit missing from the interconnect that sends the data bus to the A2 board and a different bit missing on the data bus on the A1 board. Continuity tests require contortionist skills and a steady hand, as one end is always blind and difficult to clip anything to (my replacement A3 pins are longer and you can get a hook on them below the connector, though). Trying to autocalibrate when two bits are missing produces a lot of errors, as you might expect, but even turning off calibration and using the emergency centre-of-range DIP switch override doesn't help diagnosis much because that same incomplete data bus does control as well as calibration. Control uses a 12-bit DAC, which was missing a bit, this was more obvious as it's a mulitplexed DAC which meant that DC offset, amplitude and other stuff all went in a strange set of steps instead of a nice ramp.
I think I need to replace those cables…

[synx508]

Now rid of the calibration errors, so back to a fully working unit, perhaps. Will it randomly restart? Find out another time, as I'm sick of the sight of it for the moment.

[tggzzz]

...I'm sick of the sight of it for the moment.

We've all been there

[synx508]

Testing begins…


video link to flickr

[tggzzz]

The last time I tried testing a sig gen that way, the test showed there was a problem. The audio started out reasonably, but over the course of maybe 500ms it faded and disappeared. Turn it off, and it repeated.

Later I realised the sig gen was fine, but my deafaid was deleting the signal. It was interpretating the signal as howlback, and suppressing it.

[Zenith]

By examining the different PCBs I've discovered that my 3314A's external serial number date in 1981 is completely unconnected to the boards within, the newest of which seems to be from 1995. This thing has had numerous board swaps and those interconnects should've been replaced ages ago..............

From that it sounds as if this thing was a pile of trouble from when it was new.

[synx508]

From that it sounds as if this thing was a pile of trouble from when it was new.

Well, a bit over an hour into testing and it rebooted itself! Whatever the root cause is, I still haven't found it but I have made an ugly NV SRAM modification and I suppose ruled that out as a cause. When it crashes and reboots it always returns with E09, which indicates that the battery backed memory has been corrupted and is starting with default values. This is what encouraged me to change the 1kx4 chips for a single 8kx8 chip with a similar standby current. I've a feeling the problem could be PCB related, maybe a via or a plated through hole that goes open at a certain temperature, I know that there's still a way to force it to reboot by pushing the PCB in a certain place. Perhaps, now I've got it back to where I started, I should check to see if reboots by pushing the PCB in that place also cause the NV RAM to become corrupt.

[synx508]

The previous owner of this function generator had replaced an octal flip-flop and the MC6800P CPU. I have re-flowed the soldering there but maybe I'll remove all the solder and replace it. The PCB seems particularly finicky about solder, which isn't usually the case with hp

[tggzzz]

Sounds like you are going to have to look at noise on the power lines, and signal integrity on the control/data/address bus.

The signal voltages should be easy to test with a long persistence scope setting.

IIRC the with 6800 you need to be careful with its non-overlapping non-TTL clock signals and of one of the hold times; after 50 years[1] I can't remember which one, but isn't there an E and a R/W signal? A good high speed logic analyser might help there, otherwise it is back to a scope and eye diagrams.

[1] come to think about it, it would have been exactly 50 years ago that I was getting microprocessor databooks and choosing which one to base my computer on. I looked at a hell of a lot of weird and limited devices (Fairchild F8 anybody, or MC14500 1 bit processor?). I whittled it down to the 8080, 6800 and 1802. The 8080 hardware was a pain, the 1802 simple but the instruction set was regular but bloody difficult for things like subroutine calls, while the 6800 struck a happy medium (with the exception of the clock inputs).

[synx508]

I haven't looked very closely at the two-phase non-overlapping CPU clock so perhaps I'll do that. I can't get a feel for why it would generally take a fairly long time for the problem to appear, then rather than appearing regularly, there's a similar interval before it appears again.
I did signature analysis on most of the computer board but it's not much use with a problem at appears roughly once every 8 billion cycles.
I've checked ESR of the power line capacitors, they seem very good indeed. I don't completely trust the power supply, other 3314A owners have claimed to fix this problem by soldering the pass transistors in rather than using the rather silly Molex type sockets that they use - they're like those white 0.1" connectors for the ±15V and semi-transparent 0.2" for 5.1V, of the type that looses its grip after four decades. It's not impossible that there's a small glitch as the common heatsink plate warms through a certain temperature, but that doesn't completely fit with the problem recurring at similar intervals thereafter.

If this turns out to be a software release that's got a flaw like the Boeing 787 that you have to reboot every 51 days…

Except it can't be that, because the interval varies quite significantly.

One person with the problem split the power supply's supervisory LM339 output with diodes so they could see which of the protective measures was stopping the 555 that drives the 5.1V buck regulator. I don't really want to do that but it's beginning to look like it might be necessary. I think it might be easier to piggyback a second LM339 over the top and use that to identify the problem - hoping that it wouldn't load anything too heavily.

[tggzzz]

I've never bothered with signature analysis, even when it was new. All it can do is indicate a subsystem/board that is grossly faulty, and gives no indication why.

Marginal signal integrity can be very sensitive to many things, e.g. temperature, PSU voltage. The trick is to spot what is close to the limit and likely to change.

After it croaks, do you turn it off or just leave it to recover and have another go? If so, there might be time for the temperature to fall and the timing to get back into the normal area?

Are there any electrolytics associated with generating signals? I once had an electrolytic which meant I could only do one test every 12 hours. Bit of a long shot.

If it is working and you play a hot air gun or hairdryer over the circuit, does the problem occur sooner? Ditto freezer spray? That might help isolate which sub circuit is problematic.

If worries about socket contacts, I'd put a tiny drop of contact cleaner in the socket,.and wiggle the component. Soldering contacts is for emergencies only!

Put scope on PSU, various clocks/enables look at voltages and setup/hold times for data and address lines.

[synx508]

After it croaks, do you turn it off or just leave it to recover and have another go? If so, there might be time for the temperature to fall and the timing to get back into the normal area?

I've generally left it running because I was hoping it would reveal more about the problem. If it happened repeatedly after reaching operating temperature I'd know it was something about being at operating temperature. If it did it once as it got to operating temperature I could assume some sort of continuity issue related to thermal expansion, maybe a cracked via or plated-through hole. But what it actually does is crash at reasonably similar intervals which are about the same as the time it takes to get to operating temperature and this makes me wonder if the thermal aspect is a distraction.

If it is working and you play a hot air gun or hairdryer over the circuit, does the problem occur sooner? Ditto freezer spray? That might help isolate which sub circuit is problematic.

If worries about socket contacts, I'd put a tiny drop of contact cleaner in the socket,.and wiggle the component. Soldering contacts is for emergencies only!

I don't have any freezer spray currently but I did get it quite warm with the hairdryer yesterday and I couldn't trigger the reboot. Worse than this, in a way, is that I did trigger a reboot with some connector wiggling but the reboot did not destroy the contents of NVRAM, so it's not the reboot I'm looking for.

Put scope on PSU, various clocks/enables look at voltages and setup/hold times for data and address lines.

I've done so much watching lines with my scope now but only once have I seen anything, which is what prompted the CMOS hex inverter to be replaced. Maybe I should look at the /CS and /WR lines on the NVRAM along with the data bus, using the logic analyser, since the bad reboot corrupts this. This is also where the 3314A's software causes an additional diagnostic roadblock by being helpful, it detects corruption on startup and restores defaults to all the NVRAM configuration memories. I know that it stores the working configuration in configuration slot 0, as the 3314A won't even start without working NVRAM. The slot 0 feature is quite nice, because it stores the settings-at-powerdown there which can be recalled with RCL 0, they don't automatically restore because, as the manual points out, this might be very bad in certain applications. It actually does more than store configuration there, it stores the full UI state, it even remembers which context the display was left in. It may do this with a "dying gasp" write, triggered by the PSU detecting that the AC power has gone away and perhaps this is a clue. The power supply does seem fine and is another clue to the mysteriously mixed vintage of this instrument as it has radial capacitors and strangely even much later revisions of the board where the NVRAM and RAM is a single chip and the 6 PROMs are just a single chip, has axial capacitors. Most pictures of the board online are completely different revisions but the manual talks about the early revisions having 6 EPROMs rather than PROMs.

[tggzzz]

Tricky, but more fulfilling than soduku/crosswords.

If it is a cut-and-shut with boards of a different vintage, that increases the chances of inter-board timing violations.

"Last gasp save" operations are likely to be based on noticing the mains or unregulated power lines are failing shortly before the computer's power lines are out of limits. That wouldn't work if the problem is between the main PSU and the computer.

The nice thing about a "proper" LA (i.e. not the 24MS/s USB things) is that you can set it up so it captures nothing until the fault occurs. That requires

• amplitude: variable threshold levels
• timing: a decent set of FSM-based filter and arm and trigger options. Looking for marginal timing violations requires high sample rates; gross timing violations might not.

Failing that, using the LA with one line on the PSU and threshold=4.75V might help.

I have an HP1682 800MS/s analyser for such purposes

[Zenith]

This seems a bit like teaching grandmother to suck eggs.

Have you cleaned all contacts and checked interconnects? They can be the cause of odd intermittent faults.

What about dry joints?

Without knowing anything about this model, I'd put a DSO on the power lines to at least rule out PSU glitches. Of course glitches may not be caused by PSU faults.

The number of board replacements sort of suggests they had problems, such as timing problems, which were never properly sorted out in R&D or by later upgrades.

[synx508]

The 5.1V rail seemed like the obvious thing to monitor and I'd already done that, but all it told me was that it was being momentarily interrupted and this was triggering the CPU reset mechanism and causing a reboot. What wasn't clear is why this was happening. Different bad states are ORed together in that LM339 that I've already replaced and I was originally planning to add a second one with parallel power and input lines and individual o/c outputs so I could see which comparator was firing. This turned out to be overthinking, so instead I watched the voltage differences with my voltmeter set to monitor highs and lows (actually used the 3456A in STAT mode, so UPPER and LOWER). The overcurrent shutdown was my first suspect, maybe a component was drawing excessive current occasionally and tripping it.
This was a bad choice, there was a lot of headroom between the operating current and the trip point and after a few hours of waiting all I got was a new LOWER reading, which was just over half the usual current. This was likely the period immediately after the CPU had been reset. I had missed the cause and seen another symptom.
I then monitored the "15V line is good" comparator and after hours it didn't trip, so I tried again today by simply monitoring the +15V regulated line with the DSO with a negative slope trigger set at around 14.8V. After 3.5 hours it tripped, I saw the DSO capture the event and watched the 3314A do its startup countdown. The voltage had ramped down from 15V to below 14V in a straight line over a few milliseconds. I am not sure why yet, but perhaps it is overcurrent or perhaps it's thermal runaway in the overcurrent circuit, or perhaps it's half the 1458 op-amp responsible for closing the regulation loop misbehaving.
The +15V rail gets its voltage reference by inverting the -15V rail, so before I went any further I repeated the test on the -15V rail with the slope reversed and a new threshold set at -14.75V. The restart happened in less than an hour, so maybe it really does happen more often when it's warm. Crucially, though, the DSO didn't trigger and the -15V rail was still absolutely where it should've been.
Everything is now pointing to the +15V regulator, so I'll check to see if there's a current spike next, but I don't think it's going to be that because of the relatively slow change in output voltage. I'll probably use both channels of the LeCroy DSO and measure across the 15V rail current shunt, which is on the unregulated side of the regulator, annoyingly. This means I can't completely separate the effects of load changes from a fault in the regulator circuit itself. I could monitor the pass transistor base drive shutdown transistor that the overcurrent protection uses, but that feels like it might also be not entirely independent and I'd need three channels to do that concurrently. Though maybe if I use the not-entirely-restored-yet Tek 468 in digital storage mode I can do it, with every chance of a HV disaster before the results are in!

[EC8010]

This is getting serious. That instrument looks old enough to use mica washers to insulate transistors. I once spent a solid two days looking for a fault in a 300V regulated supply and it turned out to be a damaged mica washer between 317 and chassis that tripped when 80V was reached. I have not used mica washers since then (use plastic and very carefully) and always test with Megger between chassis and appropriate pin after bolting device to heat sink before adding any wires. Just a thought...

[synx508]

This is getting serious. That instrument looks old enough to use mica washers to insulate transistors. I once spent a solid two days looking for a fault in a 300V regulated supply and it turned out to be a damaged mica washer between 317 and chassis that tripped when 80V was reached. I have not used mica washers since then (use plastic and very carefully) and always test with Megger between chassis and appropriate pin after bolting device to heat sink before adding any wires. Just a thought...

The insulating washer is a good call, though in this case it's using silicone washers and there's less than 40V between the collector and chassis. I don't know if these are original. +15V rail's current shunt resistor voltage drop is indicating that the load is drawing 400mA, I've got the 3456A checking the voltage across the shunt resistor right now. It's a shame that it doesn't seem to be possible to combine maths functions on the 3456A, I can either have stats or I can directly compute the current by telling it to divide by the shunt resistance.

[tautech]

Old beaded resin dipped Tants can have a fault condition called sputtering but it's quite rare.
The tantalum momentarily breaks down but self heals but in the time to do this might bring the rail low enough to activate the reboot.

If there are any in this unit, replacing them might eradicate the frustration.....just be sure to increase their voltage ratings.

[synx508]

Old beaded resin dipped Tants can have a fault condition called sputtering but it's quite rare.
The tantalum momentarily breaks down but self heals but in the time to do this might bring the rail low enough to activate the reboot.

If there are any in this unit, replacing them might eradicate the frustration.....just be sure to increase their voltage ratings.

I don't think there are any tantalum beads but there are tantalum axial capacitors, the computer/PSU board has two 15µF 20V on the 15V rail and there a few 2.2µF 20V on +15V on the other boards.

There has just been another restart and the highest current measured on the +15V rail was 410mA with the normal operating current being 397mA. It's possible that the peak was too short for the relatively low sample rate of the 3456A. If the service manual mentions a typical current then I've not noticed it yet. Maybe I'll change these capacitors…

[Zenith]

As well as tants with their faults, I have noticed that electrolytic capacitors can break down, cause a glitch, and then reform almost immediately and carry on for another half an hour. I've spotted it with hi-fi amplifiers and old radios.

The usual and sensible advice is to make sure that power supplies are solid before delving further. It doesn't really encompass PSUs with transient faults.

Can you separate sub systems out? Remove the inbuilt PSU from the problem and use a bench PSU or two. Use the inbuilt PSU to power equivalent loads and monitor what it does.

I'm making a mental note not to mess with an HP 3314A function generator, unless I come across one for a fiver or less.

[tggzzz]

I've had a couple of axial tents spew acid on a board, fortunately only 2 layer.

Electrolytics are weird beasts, with some very transient fault behaviours.

Are there any other electrolytics near any of the control lines for the PSU/protection circuits?

Other caps can have unpleasant internal faults, cracking, whiskers (I guess). Tried flicking anything with your fingertips?

[AVGresponding]

It's also not that unusual for crapacitor faults to only show up at or near the working voltage, which can make testing them with the average LCR meter prove to be non-indicative.

[synx508]

I've had a couple of axial tents spew acid on a board, fortunately only 2 layer.

Electrolytics are weird beasts, with some very transient fault behaviours.

Are there any other electrolytics near any of the control lines for the PSU/protection circuits?

Other caps can have unpleasant internal faults, cracking, whiskers (I guess). Tried flicking anything with your fingertips?

HP 1725A,1715A and 1740 series oscilloscopes all have a 100µF wet tantalum in their first timebase that eats its own negative lead, then the acid leaks out and consumes the track that connects it to the timebase selector switch, the timebase selector switch and the spring detent mechanism for the timebase selector. But there was no sign of leakage on the two axial tantalum capacitors in the immediate vicinity of the PSU. I've changed the capacitor responsible for +15V but I have dropped from 15µF to 10µF and increased 20V to 25V. This seems to have upset the "last gasp", if indeed it has a last gasp system, at any rate, it seems to have affected the NVRAM's reliability, so maybe they used a 15µF with 10% tolerance for a reason?

There aren't any electrolytic capacitors involved in control timing, the largest capacitor there is a ceramic 100nF in an unusual orange resin-like finish, it looks like those Philips tantalum capacitors, but it's a ceramic. I've already established that the fault is a ramping down of the +15V rail rather than anything abrupt. That's one of the reasons I think the attention to capacitors is not going to lead to a solution, but given that it has already lead to an apparent worsening of the "resetting to defaults" condition through NVRAM corruption perhaps that is wrong. Maybe the whole PSU is a cunningly disguised analogue computer that controls the digital computer through a series of voltage and current ramps. That's certainly how it feels. Bad NVRAM data doesn't always get detected on boot, interestingly. I've witnessed negative sweep times and byte values that don't result in conventional BCD decoding on the display but can still be incremented and decremented until the value reaches the programmer's intended bounds checking! I actually love stuff like that and I have disassembled the ROM in the hope that I might get around to adding a memory map so the disassembly makes sense etc. There's a lot of code though, 64kiB is such a huge ROM for 1981. Then there's 1kiB RAM and the same again for NVRAM. Considering this is a half rack width machine it contains a fairly hefty computer and the analogue performance is also way, way better than something like a 3325A, this can make all its waveforms across the whole frequency range.

I've been gently battering the PSU part of the circuit with pens and the handle end of screwdrivers with no response. I've had reboots by blasting the hairdryer at it, but they're not consistent or repeatable. Elsewhere on the A3 computer/psu board, the interconnects are behaving a bit better but they're not completely right and I've seen "E51" a few times, which is the error you get if someone has been feeding high voltages into the output and the internal series protection relay has opened as a result. Continuity tests both for what should be happening and for shorts between adjacent pins always show the interconnects to be absolutely fine, because of course they do. It doesn't even have the properly cursed interconnects from the later 3314A and the 3325B, those flat printed ribbon cables that transition to pin header style connectors at the very end and go gooey on the ribbon part and intermittent on every other part. These earlier 3314As have almost conventional wire based ribbon cables, there's even a key and it's even in different places on different connectors so you can't get that wrong easily. This is almost unheard of on HP gear and it makes me wonder if the 3314A is the work of "that new guy".

It's an excellent machine, I'd have given up a long time ago if there was anything else that does as much in a smallish box, even compared to modern fast DAC based arbitrary waveform generators it can do lots of useful stuff rather well.

[synx508]

It's also not that unusual for crapacitor faults to only show up at or near the working voltage, which can make testing them with the average LCR meter prove to be non-indicative.

I tested the big capacitors by removing them and testing with the LCR meter, then testing by putting them on a PSU with the voltage set a little above where they'd be operating and seeing if anything odd happened, then finally by running the 3314A with alternative capacitors and noting that the capacitors weren't the problem. I do have a home-made ESR tester, so I might add that to the list of tests.

There are more capacitors to test, however… I reckon it's a transistor or diode in the 15V regulator circuit rather than something on one of the generator or mode boards.