The Super Cold Covid Vaccine Distribution Problem

Even though Mr. Market is over the moon about the prospects for a not-yet-approved-by-anyone Pfizer Covid vaccine, Naked Capitalism readers quickly pounced on the question of whether or not the unprecedented mass distribution of a vaccine that needs to be kept at roughly -100F for no more than five days before use might be a problem. As vlade underscored by e-mail, “I don’t think that anyone was expecting having to distribute literally billions of items that have to be held at -80C or less until earlier this year.”

The mainstream media has caught up with reader concerns. As we’ll discuss, building new super-cold distribution is so costly and daunting that developing economies are likely to turn down the Pfizer vaccine. And not only are rural areas expected to get the short shrift, but even US states are having trouble developing credible distribution plans, which suggests that even some cities may encounter delays in getting their hands on the vaccine (assuming it is approved quickly).

Keep in mind that the same distribution issues apply to another promising-looking vaccine, from Moderna, which is another messenger-RNA-based vaccine and also must be kept at very cold temperatures before use, although not as severely cold as the Pfizer candidate.

And yes, sports fans, no such cold-chain storage system exists now, anywhere in the world. Keep in mind that the problem isn’t just keeping the vaccine vials super cold till they get to where they will be used. They also need to be kept in a super-frigid state at close to their point of delivery, or delivered on a just-in-time basis.

ProPublica gives a good overview of the delivery and storage issue:

The Pfizer vaccine is unusually difficult to ship and store: It is administered in two doses given 28 days apart, has to be stored at temperatures of about minus 100 degrees Fahrenheit and will be delivered in dry ice-packed boxes holding 1,000 to 5,000 doses. These cartons can stay cold enough to keep the doses viable for up to 10 days, according to details provided by the company. The ice can be replenished up to three times. Once opened, the packages can keep the vaccine for five days but can’t be opened more than twice a day. The vaccine can also survive in a refrigerator for five days but can’t be refrozen if unused.

Note that this description focuses only on the requirements. Below is a graphic in the Wall Street Journal as to how Pfizer plans to pack its vaccine. Clive saw a presentation and just sent this e-mail:

BBC ran this last night (think Pfizer went on a PR offensive over it) https://www.bbc.co.uk/news/technology-54889084

Two things struck me: one is insulated boxes are only currently available certified for storage down to -8C (so not suitable for low temperature medicines which need around -18C, but probably a good option for medium temperature cold storage). The Bloomberg (and the WSJ article which lead coverage) referred to Pfizer’s solution being capable of maintaining -70C (so they’ve already been bartering themselves down from previous statements saying the vaccine needed -80C — I’m always on alert when what should be absolute technical details start getting fuzzied). They mention dry ice as a source of thermal absorption, which without built-in mechanical cooling on the storage box can’t reach down to -80C. Nor do Pfizer mention any testing or product certification having been carried out. The timeframe for maintaining box temperate was also vague — it will have to have, by the laws of thermodynamics, different performance curves in high ambient air temperature regions for instance and it wasn’t spelt out if the quoted temperature maintainability time was relying on being in placed either a medium or low temperature refrigerated environment.

The other thing Pfizer ‘fessed up to was the phials were only capable of withstanding 4 exposures to ambient air without compromising stability. A logical guess is that embrittlement of the pharmaceutical-grade seals on the phials caused by any repeated thermal stress eventually means there’s soo much risk of allowing any more. So, four strikes and you’re out. You can envisage one exposure to ambient air as the vaccine leaves the the vaccine finishing plant and gets packed for initial transportation to the bulk storage facility. Another exposure will occur when they’re unpacked and placed into bulk storage. Then another exposure when they’re lifted out of bulk storage and repacked for local distribution. There’s one “life” left after all that — unpacking and thawing prior to dispensing. So zero room for any errors or unintended ambient air exposure.

For a Big Pharma player who must know there’s going to be a lot of scrutiny about quality control and vaccine safety, I don’t get why they think vague musings to tame press outlets and “tah-dah!” hero techie fixes like the storage box are going to cut it.

As Clive and vlade discussed by e-mail, there’s a big “last mile” or perhaps more accurately, “last stop” problem. What happens if many vaccine injection sites can’t be put on “just in time” deliveries? Then the vaccine would need to be stored locally at super cold temps until it’s time for it to be thawed and used. Vlade and Clive discussed the requirements. First from vlade:

The low temperature sites _are_ in fact farms of big boxes like these https://cryometrix.com/product-t160 . This is specifically meant to hold 2ml vials like medical samples or vaccine doses. They are not really that hard to have, a neighbor of mine is in the cryo/liquid gas business and embryo/stem cells long term storage is one of the areas he does.

A walk-in vault ala meatpackers is just not viable at -60C.

Clive replied:

Yes, all commercially-available ultra-low temperature boxes are reach-ins. I’ve never come across an ultra low temperature walk-in, nothing above a small cupboard size, anyway (i.e. just a slightly bigger reach-in with a separately-installed second-stage split evaporator). So it makes sense to get enough of these reach-ins together and network them to act as a static bulk-storage facility. They’re sill a niche product though, and while you can probably source them in the hundreds, if not from stock, then with fairly short-ish manufacturing lead times, trying to get hold of thousands (which would be needed for continental-scale operations) isn’t going to be easy. Even if manufacturing / production capacity was available, it’ll be the sub-components — especially compressors, lubricants and refrigerants — where the bottle necks will be.

And they (reach-ins) will be under more supply pressure because in addition to bulk storage applications, they’ll also be needed for district and local storage (county and city hospitals, local health centres and so on).

Bloomberg gives a higher-level picture of why many developing countries are set to give the Pfizer vaccine a pass:

When Pfizer Inc. and BioNTech SE’s Covid-19 vaccine rolls off production lines, Shanghai Fosun Pharmaceutical Group Co. will be waiting to distribute it through a complex and costly system of deep-freeze airport warehouses, refrigerated vehicles and inoculation points across China…

….ountries will need to build from scratch the deep-freeze production, storage and transportation networks needed for the vaccine to survive. The massive investment and coordination required all but ensures that only rich nations are guaranteed access — and even then perhaps only their urban populations.

“Its production is costly, its component is unstable, it also requires cold-chain transportation and has a short shelf life,” said Ding Sheng, director of the Beijing-based Global Health Drug Discovery Institute, which has received funding from the Bill & Melinda Gates Foundation…

“If there is a protein-based vaccine that could achieve the same effect as an mRNA vaccine does and there’s the need to vaccinate billions of people every year, I’d go for the protein-based shots in the long run,” Ding said….

“The requirement for extremely cold temperatures is likely to cause spoilage of a lot of vaccine,” said Michael Kinch, a vaccine specialist at Washington University in St. Louis.

The article discusses how India is just about certain not to deploy the Pfizer Covid vaccine. It barely has the capacity to handle measles vaccinations, which target a smaller population (literally and figuratively), children aged 3 or below, and required cold but not super cold storage. Another impediment is the need for mass training of technicians to make the injections.

Given that the Pfizer vaccine requires two shots, a month apart, how often will patients wind up getting one or more jabs with vaccine that’s gone bad, and think they have Covid protection when they don’t? And in rural areas, where people often have to travel to an injection center, they often skip the second shot.

Back to the US. The Wall Street Journal says Pfizer will handle its own distribution, while in the US, McKesson will manage the distribution of other vaccines.

However, the ProPublica story, on state plans, demonstrates that “distribute” may not mean “deliver to hospitals and big clinics” but appears to mean “deliver to state public health officials who decide what to do next”. In reality, I have to think that major health providers like hospital systems would deal directly with Pfizer, but even so, there looks to be tremendous variations at the state level as to what the drill will be. ProPublica focuses on the problem of getting the vaccine to rural areas, but its survey of state plans raised additional questions.From ProPublica:

As the first coronavirus vaccine takes a major stride toward approval, state governments’ distribution plans show many are not ready to deliver the shots….

Across the country, authorities are grappling with how to accommodate the Pfizer vaccine’s finicky specifications. So far, state plans show few have come up with clear solutions. Oregon, for instance, said it still needs to “develop [a] plan” for how to handle 1,000-dose orders in “remote Oregon locations, while maintaining the ultracold chain and avoiding wastage.”

Perna, the general leading logistics for Operation Warp Speed, told NPR that it’s up to states to buy more freezers. That contradicts the CDC’s instruction to not invest in more equipment. But many states said they’re doing so anyway, or at least looking into it. They’re also taking stock of what facilities already exist in their states at hospitals and universities, or where they can get dry ice.

North Dakota, where the virus has killed roughly 1 out of every 1,200 people, is considering whether to break down the 1,000-dose packages and, on its own, distribute smaller quantities to individual hospitals and clinics….

As Texas and other states consider the need to break down the 1,000-dose packages into smaller shipments, that’s an additional cost that they’d have to shoulder, since the federal government will only pay to move the vaccines once. Several states identified funding as a major problem.

Virginia’s plan included a “preparedness gap analysis” that estimates that it will need $71 million to establish and operate mass vaccination clinics, which would include hiring temporary staff and covering facility rental costs, translation services, signage and other operating costs.

The plan also calls for a further $2.5 million in equipment such as refrigerators and thermometers and $3 million for public education, including TV, radio and social media ads, as well as “targeted outreach to clinicians, vulnerable populations and other key groups.”

On the other end of the spectrum, Kansas simply assumes Pfizer will indulge them by shipping minimum dose sizes of 100 per order. The story also points out that expecting rural doctors and nurses to travel to get shots is costly, supporting the Kansas view that the Pfizer minimum size is way too high:

Even in the case where prioritized health care workers were physically capable of driving themselves to the city to get a vaccine, relying on doctors and nurses to get themselves to a vaccine “doesn’t compute,” said Tim Size, executive director of the Rural Wisconsin Health Cooperative, which represents 43 rural acute hospitals. Wisconsin is battling its worst outbreak of the pandemic, and every hospital is stretched thin on staff, he said. Requiring everyone to take time off, twice, to get the Pfizer vaccine “means two days of lost staff time at a time we’re desperately short of staff.”

Despite the Pfizer happy talk about the possibility of starting to ship the vaccine by year end, that assumes everything breaks its way. But even on a fast track, the roll-out will be largely or entirely a Biden Administration task. We’ll see soon if it lives up to its campaign promise of competence. While the Trump Administration set a very low bar, elite symbol manipulators and complicated real world situations with hard physical constraints are not a happy mix.

Print Friendly, PDF & Email

72 comments

  1. marcel

    The russian ‘Sputnik V’ vaccine is announced to be 92% effective, and does not require such cold storage. But you can’t have nice things, especially not from Russians.

    Reply
    1. The Rev Kev

      I read that Hungary at least was thinking of using Sputnik V and then the EU stepped in and said ‘Oh no you don’t. We haven’t tested it so we think that you can’t use it. Doesn’t matter that we have no plans to test it ourselves.’

      I just read now that Hungary will go ahead anyway and use it for testing and licensing. With the logistics problems enumerated here with the Pfizer vaccine, we may have to go with the Russian vaccine in the west. If it is effective that is.

      Reply
      1. timbers

        If the Russian vaccine has about the same success rate claimed by Pfizer but without the cold storage issue, makes sense to use Russia’s vaccine. But I don’t follow Covid medical news like others here do.

        The CIA must have assets in the area that can steal the Russian formula (I’ve watched the Bourne movies I wasn’t born yesterday).

        Reply
  2. Stephen The Tech Critic

    > Given that the Pfizer vaccine requires two shots, a month apart, how often will patients wind up getting one or more jabs with vaccine that’s gone bad, and think they have Covid protection when they don’t?

    I am far from an expert on this subject, so please take this with a grain of salt, but I would also ask whether injection of spoiled vaccine might actually cause harm to patients.

    I recall learning some time ago that a major problem with many protein-based is the potential for protein misfolding, where the protein does not achieve or maintain its correct 3D shape. These misfolded proteins can cause considerable harm to the subject in addition to being ineffective. I believe mRNA (which this vaccine is made of) is essentially a helical string rather than a complicated 3D structure, and therefore doesn’t have the problem of misfolding. However, if something causes corruption to the information encoded in the base pairs (the “code”), the proteins that are manufactured from them could be something entirely different and unexpected. I think that’d probably be bad.

    Reply
    1. Yves Smith Post author

      I was concerned about that too, but didn’t want to raise possible false alarms in the post, so glad you brought it up. Hope readers with relevant expertise can weigh in. There may be indicators or warnings that might allow for most vials of spoiled vaccine to be caught, but this is way above my pay grade.

      Reply
      1. Internal Medicine Doc

        Yves,
        Medical doctor here.
        I was on a web based live grand rounds from my Alma mater yesterday- one of our great research institutions.

        The speakers were a Pfizer supported virologist, an immunologist and a biochemist.

        The freezing issue was glossed over. “Yeah that’s an issue but we will figure it out somehow”

        But a very relevant question related somewhat to your commenter above was the following:

        Multiple medical diagnoses are in a family called paraproteinemias. The classic one being multiple myeloma. They are blood cancers where the malignant cells make proteins – in the case of myeloma -immunoglobulins. Over a period of time this gets out of control and the proteins themselves begin to damage the kidneys and blood vessels severely.

        Since we are proposing to inject into either epithelial cells or immune cells mRNA that will hijack their ribosomes to make covid spike protein – and we would presume this will then replicate as the cells divide – how do you experts propose that this process gets turned off? What if some patients that are vaccinated with this eventually flood their body with spike proteins similar to a myeloma patient? How often do you believe that would happen? Is this even a concern?

        Answer from the three experts – CRICKETS CHIRPING.

        I have found your commentariat highly intelligent and resourceful. Maybe one of them knows the answer to this excellent inquiry.

        This is what I do know – myself or any of my patients will be getting nowhere near this vaccine until there is complete transparency. With all the anti vaccine voice out there, screwing this up may be the death knell for vaccine science and even possibly medical science.

        I agree with Lambert. The first 535 test subject should be the US Congress. That sounds glib- but when you really think about it , it makes sense. Only they are going to have the ability to cut through the crap.

        Reply
        1. David Utzschneider

          The mRNA does not become part of the host genome, it eventually just degrades and so the encoded proteins are no longer made. Like the message at the beginning of the Mission Impossible movies, you have a brief period of time to listen, and then it’s gone

          Reply
          1. Internal Medicine Doc

            That has been the big anti vaccine myth that the rna goes permanently into the genome

            That is absolutely not true it never ever enters the nucleus so that really is a myth.

            However, based on what I am hearing from experts I trust it does seem that in some animal models the mRNA is more permanent than you describe above.

            The question I reported above would never have been asked in that kind of setting if there was not some genuine concern.

            Reply
        2. Kurtismayfield

          Dear medical doctor:

          The reason why a myelnoma patient can spoke proteins because the mutated gene from the cancer cell is in their genome. We are talking about an RNA sequence that is added to the body. The concentration will only degrade over time, and the cells cannot increase the amount of RNA.

          Reply
    2. Stephen The Tech Critic

      And here I embarrassed myself—because mRNA doesn’t have base *pairs* like DNA does. It’s just a single strand. Sorry!

      Reply
    3. Phacops

      While mRNA is no protein, you are right about protein structure and immunologic response. that is why Pfizer’s B1, containing only the spike protein active region, failed. The structure of that small peptide probably did not conform to the structure found in the complete protein. The simplest explanation.

      The next simplest explanation is even more fascinating (I’ll leave out cytokines, and specific molecule names). When a virus enters a cell it gets chopped up into smaller peptide subunits. These bind to a histocompatibility molecule that migrates to the cell surface. That complex is picked up by helper T cells which interact with B cells causing them to mature into two classes, plasma cells that secrete antibody, and memory cells that are “trained” to recognize the antigen. Those memory B cells, if contacting the antigen again will process it and re-present it to the T cells.

      Pretty interesting stuff demonstrating how vaccines are intended to stimulate immunologic memory.

      Reply
    4. Olivier

      mRNA also folds, just not in shapes as complex as proteins. Mercifully there doesn’t seem to be an mRNA equivalent of prions, though.

      Reply
  3. TMoney

    This detail was news to me -> a vaccine that needs to be kept at roughly -100F for no more than five days before use.

    Even if the cold distribution problem was as easy as my earlier ignorant remarks (Its not) suggested, Knowing you only have 5 Days to get this from the plant to the patient – Loading & Unloading, moving to warehouses moving to hospitals and having patients waiting when it arrives. – well doing all that with such a short clock requires an effort that even my fertile imagination can’t comprehend.

    Reply
    1. Yves Smith Post author

      I think you mis-interpered the sentence. The vaccine can presumably be kept frozen for months. But once it is thawed, it can be at refrigerator or warmer temps only for 5 days.

      Reply
      1. KLG

        This is indeed the case. I’ve been a working biochemist/molecular cell biologist since before recombinant DNA (cloning) was a thing. Many reagents are stable long term only when frozen at -80C, but can remain active for a few hours/days in a household freezer (-20C) or when refrigerated (0-4C). A few reagents and virtually all “live” vertebrate cells must be stored at -196C (in liquid nitrogen). Having sent and received shipments on dry ice hundreds of times each, this is an absolutely fatal flaw for a biologically active human therapeutic, absent a military mobilization described below by Clive. And even then, if those giving the shots are not those who break the seal on the package from the manufacturer (with an interior still at ultracold temperature), how will they know the vaccine is still functional?

        Reply
        1. KLG

          And about those freezers, as noted below by Yves they are very expensive (low end at the cost of the -20C machines used by Duchess Nancy of Pacific Heights to store her Dove bars) and prone to failure. Especially those conveniently wired for 110V AC instead of those wired for 220V DC. A room full of these things also requires a separate HVAC system for cooling, since they put out a lot of heat, which makes the compressors work all that much harder if not dissipated. Supply is also very limited, with distribution through scientific supply houses. You don’t get these things at Home Depot or Lowe’s.

          Reply
          1. petal

            I purchased our -80 freezer 7 years ago(yikes time flies) for ~$12,000 USD. A few months ago I purchased a new -140 freezer for $19k USD. Lead time was at least a month. Friends and colleagues regularly have issues with the stability of their -80s, and -140s are notorious for problems. Our previous -80 and -140 had repeated compressor problems as KLG talks about. Repairs are very expensive and take time-if parts are even available. I think not long ago I heard through the grapevine -80s and -140s are now in short supply.

            Reply
            1. KLG

              A “rich” institution has one empty -80C freezer for every 4-5 that are full…spare for the inevitable. I can’t count the number of times (on a Saturday night when the alarm called the campus police who then called me) I have scrambled to move irreplaceable items from a warming freezer to the emergency spare or into a friendly colleague’s freezer that is not full (good luck with that; not the friend, the half empty freezer). If this thing requires this level of frozen chain-of-custody, it. will never work. Pfizer can build all the freezer space it wants to, but we cannot drive to Kalamazoo for our shots…And as Ignacio notes, RNA is notoriously prone to degradation by RNases that are everywhere. Everywhere. Half of all molecular biologists seem to be unable to work with a coding mRNA at all. As a mostly linear molecule it is doubly susceptible to degradation. tRNA and rRNA have secondary and tertiary structures that can stabilize the molecules. A scientific worker is either born with hands that allow her to handle RNA, or not. This seems to be a marginally teachable skill.

              Reply
  4. Brick

    Most vaccines are in transit from the manufacturer to the patient for more than 10 days.Most drug wholesalers will be ordering on a weekly schedule and hold roughly 12 days stock. From Factory to the wholesaler you can have ships, aircraft and wagons delivering with limited refrigeration capability and probably no fragile box protection. From the Wholesaler to the Pharmacy or Hospital is usually by van again with certain limitations on capabilities.
    This basically means the usual wholesale route to market is out and the logistics need to be handled either by state parties or by Pfizer.The thing that struck me though was the 1000 to 5000 doses which are too many for your average pharmacy or Doctor’s to get through in the timescales required. So it has to be administered by hospitals (who are too busy) or special centers.
    The chances of getting a spoiled vaccine seem worrying( State run with maybe inexperienced personnel) and I suspect a lot of people will wait for the next vaccine.

    Reply
    1. Clive

      The Uk government has mooted that the armed forces will be needed to mobilise a vaccination programme:

      https://www.theguardian.com/world/2020/nov/10/covid-vaccine-ready-december-rollout-matt-hancock

      The UK health secretary indicated he anticipated a “mass rollout” would happen in the first part of next year if the vaccine was given the green light, although he said he had requested the NHS, supported by the armed forces, to be ready from the start of December.

      Priority for the vaccine will go to people in care homes, elderly people and social care staff. It will be delivered through care homes, GPs and pharmacists, as well as “go-to” centres set up in venues such as sports halls.

      “We will be working across the NHS with the support of the armed forces seven days a week, over weekends, over bank holidays, to get this rolled out into people’s arms as quickly as possible,” Hancock told BBC Breakfast on Tuesday.

      Note that, while there’ll be some outreach to less mobile populations, the “go-to” centres will presumably be responsible for the mass roll out. As you say, primary care and pharmacies simply don’t have the capacity in and of themselves — and if supply is in multiples of thousands of doses, they have neither the capacity to store it nor the personnel e.g. nurses to use it anyway. And there’ll be no slack in the supply chain capacity and vaccine availability to tolerate a lot of spoilage losses.

      Oh, I must of course mention the UK government couldn’t organise a piss up in a brewery, so take any pronouncements by the Health Secretary with a bucketful of salt.

      Reply
    2. rd

      Defense Production Act

      My understanding is that there are at least two candidate vaccines that need similar refrigeration. We know what temperature they need to be kept at and we have a pretty good handle where and how it would be distributed. We also have a projected time frame of potentially 5 months from now. Manufacturing facilities can start gearing up to produce refrigeration equipment appropriate for fixed sites and mobile transport units. The federal government picks up the tab. Far, far cheaper than economic impact of another year of waiting for natural herd immunity.

      However, this Administration and Congress will assume the private sector will develop the successful approach over the next five months and be positioned to deliver the vaccine in a massive countrywide one-off event. If our modern government was in place December 7, 1941, George C. Marshall and Chester Nimitz would have had to start Go-Fund-Me web pages to fund the construction of new aircraft carriers, planes, and tanks.

      Reply
  5. Synoia

    Faced with a difficult distribution problem, we do not have a usable Vaccine.

    What temperature is required to make the Vaccine? Is make locally and inject immediately a possible process?

    Can a vaccine making machine be mounted on a truck?

    Reply
  6. Bob

    How is the distribution of this vaccine different than the commercial distribution of material for artificial insemination used in animal husbandry ? This amounts to large numbers of doses.

    Let’s not get ahead of ourselves – there are issues with low temperature distribution however these are real issues that beg for real solutions.

    Reply
    1. Yves Smith Post author

      Huh? Surely you know that veterinary medical approaches can’t be applied to humans without lots of studies and approvals. And the sources and end points for artificial insemination of animals are completely different than for humans needing the Covid vaccine.

      Animal artificial insemination uses a nitrogen cold chain. As Clive said yesterday:

      Repeat after me: Liquid nitrogen alone is not certified for medicine storage and distribution. Liquid nitrogen in just any old container is similarly not certified, either.

      Storage of medicines and approved storage solutions is subject (at least in the EU and I’d have thought the FDA take a similar approach) to strict regulations. You need both temperature sensors (with redundancy) and data logging and reporting to certify stable conditions, plus an alarm to alert staff for any temperature excursions.

      There are a small number of ultra low temperature freezers which are suitable for medicines storage. Here’s the entry level https://labcold.com/wp-content/uploads/2020/06/LULT3585.pdf, suitable for a small primary care practice or local pharmacy. The sticker price? £8,365 (so that’ll be around $10,000 US) https://www.pharmacy-equipment.co.uk/refrigeration/laboratory-freezers/ultra-low-freezers. Something a little bigger, suitable for, say a hospital or, in multiple combinations, a vaccination centre? Nearly £10,000. A basic -80C freezer without the required temperature monitoring? £6,000 (ultra low temperature probes and data logging needed on top of that).

      Just how many of these units are there out there at the moment, in medical facilities? What’s the lead time on factory orders? How quickly can production be increased — these need specialised refrigerants, compressors and compressor lubricants, plus suitably robust insulating materials.

      These are all storage solutions. Transporation is a whole other matter. As is medicine handling and preparation for administering doses.

      Pfizer has published no protocols on any of these things. In the absence of an approved set of protocols, you can’t specify the potential storage and distribution methods.

      The reasons Pfizer hasn’t published protocols is that the current EMA medicines testing regulations do not cover, in their off-the-shelf form, medicines that need storage below -20C. These need, to quote https://www.ema.europa.eu/en/documents/scientific-guideline/ich-q-1-r2-stability-testing-new-drug-substances-products-step-5_en.pdf

      2.1.7.4. Drug substances intended for storage below -20°C

      Drug substances intended for storage below -20°C should be treated on a case-by-case basis.

      So, someone will need to define, document and approve this particular use-case for this particular medicine.

      Reply
      1. Bob

        No.

        The point is that the procedure for low temperature commercial distribution of material for artificial insemination in animal husbandry is effective and widely used. In addition it should be noted that it has become common for both sperm and eggs to be frozen and stored for human use.

        Ego while medicines that need storage below -20C will require special protocols, it is possible to develop such protocols that meet the the requirements for human consumption.

        It is not as if the proposed vaccine is the first to require or use of low temperature material for medicine. This is not a new nor an insurmountable issue.

        Reply
        1. Clive

          Please, not the bull semen again. I dealt with this below in my comment timed 7:55 Eastern.

          There are completely different standards for animal / veterinary products and procedures than there are — rightly so — for human medicines. You cannot take a position which says that we’ll apply veterinary regulations to human vaccines. And human tissue is similarly a completely different thing than medicines, with an entirely separate regulatory regime. Human embryos (and tissue for transplants etc.) are not medicines and can’t be treated as such.

          And have you stopped for a minute and stepped back to consider what it is you’re actually proposing there? You are saying that you’ll downgrade the regulations and product standards for human medicines to that of animal fertilisation material for no better reason than mere expediency. Do you genuinely want to have that as the message to the public? Really? It will be giving a field-day to the nuttiest of the anti-vaxxer nut jobs.

          The COVID-19 Vaccination programme needs its levels of public acceptance and adoption nurtured and assisted. It does not need crappification. Clinicians, key opinion leaders and all of us who would be advocates for sound public health policies know what we have to do — and it isn’t a particularly easy task right now.

          Shipping a vaccine under a storage and distribution method used for livestock is about a counter-productive a suggestion as it is possible to think of. What’s the marketing / PR tag-line supposed to be? “If it’s good enough for Daisy the cow, then it’s good enough for me!” ???

          Reply
        2. vlade

          I don’t know enough about the veterninary market, but I believe the size of the CV vaccination is much much larger than that, and the time requirements are different too.

          As in a requirement to get literally high tens/low hundreds of millions of people vaccinated in relatively short time, twice.

          Reply
      2. Internal Medicine Doc

        Yves
        The only physician office I know that would have even a remote need for this type of freezer is fertility clinics to store sperm and embryos

        At the doctors lounge yesterday absolutely zero out of dozens of docs had this in their office

        I cannot imagine there being any kind of enough of a supply of these freezers or the manufacturing ability to get this up to usability anytime soon

        Reply
    2. Clive

      In the EU, veterinary regulation and standards are an entirely different criteria (https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:01988L0407-20111101) with a completely separate regulatory body (the EMA isn’t responsible for veterinary medicine) to human medicines. I’m not sure about the FDA, but I’d be amazed if it didn’t similarly have different regulatory responsibilities and approaches for human medicines and veterinary products. You can’t, then, lift-and-drop veterinary regulations and requirements into medicines. And put more simply (and crudely), if a cow dies because of contaminated fertile material, who cares? I’m not a cow.

      Reply
      1. apleb

        Just like the regulations were thrown away for developing the vaccine, the same can be done with distribution if they want to. No one will care, no one.
        What they cannot regulate away are actual distribution en masse and handling.

        And yes, there will be errors, with the number of doses in billions, MANY MANY errors. Ask any hospitals how many people die each year in them which actually shouldn’t. Staggering numbers.

        Btw: if a cow dies, depending on type of cow, that’s a big loss too. Probably worse for some bigag concern than losing a worker….

        Reply
      2. Phacops

        At least for the manufacture of veterinary drugs, practices outlined in CFR Title 21 Parts 210 and 211 applies.

        I have not seen details of the Pfizer container/closure system and these can be problematic under cryogenic temperatures (below the glass transition temperature of elastomers), but this has been well studied. See an example; https://www.pda.org/pda-letter-portal/home/full-article/cryogenic-storage-challenges-for-container-closure-systems. I did not see any extractable data (materials able to be extracted from the stoppers by the product), but the manufacturer must test that and seal integrity as part of the CMC (Chemistry Manufacturing Controls) submission to the FDA.

        People rarely see all the work that goes into a New Drug Application.

        Reply
        1. Howard Beale IV

          Pfizer is building a US football-sized cold storage facility for the vaccine in Kalamazoo, MI (former HQ of Upjohn, which Pfizer acquired decades ago).

          Reply
    1. Yves Smith Post author

      1. 8000 doses for a limited geographic area bears NO comparison to the current situation where hundreds of millions of vials are planned to be distributed. Did you miss that most developed countries are deeming the distribution requirements to be too daunting? These are countries that on average are more developed than Sierra Leone.

      2. The Ebola vaccine could also be stored at refrigerator temps for 14 days (see page 51), unlike the Pfizer vaccine, where the max is five days. Big big difference here too.

      3. The Ebola vaccine was also a single dose vaccine.

      Reply
    2. vao

      It seems that your sentence about the Ebola vaccine has disappeared, but just to give an idea of the magnitude of the scale that has to be reached with a SARS2 vaccination campaign:

      The Ebola vaccine was delivered and applied only to selected people — health personnel and “rings” of persons at high risk of infection because of possible contacts with provably sick people. By June 2020, when the Ebola outbreak was declared over, 303000 persons had been vaccinated in a period of two years (60000 were health personnel). This was the result of a coordinated programme involving the WHO, local health systems, pharmaceutical firms, donor governments and NGO — with matching funds and a systematic contact tracing scheme.

      With SARS2, we would have to deliver hundreds of millions to billions of doses and mass-vaccinate people within a year or so. This posting shows that there is not even a coordinated approach within the USA, that funding is still very much unclear, and as far as contact tracing goes, most supposedly “developed” countries have been completely overwhelmed.

      I doubt that approaches requiring ultra-freezing medical products will scale up.

      Reply
  7. The Rev Kev

    Thinking this problem over, I can see converted aircraft, ships and trains for example being used to ship vaccine supplies to various regions around the United States for example. But the problem remains of how to distribute these frozen supplies when they get there as you have only a window of a few days to get this done. I am probably wrong here but is all this a variation of the Last Mile Problem?

    https://en.wikipedia.org/wiki/Last_mile_(transportation)

    Reply
  8. verifyfirst

    Seems like there are way too many “trust me” pieces to this new shot–in particular, the cold storage chain of custody from manufacture to my arm–if the shot loses adequate cold levels somewhere in the distribution chain, will there be any way for the me, or the person giving the shot, to tell?

    Reply
  9. George Phillies

    The Moderna storage temperature is -4 F, or so I read. There are quality ice creams with storage temperatures lower than this, or so I read. Some work will be needed on quality control and logging, but many supermarket chains have the equipment.

    I seem to recall that the old pneumonia vaccine had to be stored colder than this. At the time I got my first shot of it, you had to go to a clinic–at least in Massachusetts–rather than a doctor’s office to get it.

    mrna in vivo degrades fairly quickly, following which production of the spike protein stops.

    Reply
  10. Carolinian

    Keep up this kind of talk and the stock market may go down–just sayin’.

    Or at least so I read. The notion that they would pursue such a ridiculously impractical “solution” suggests that it’s not just Trump who is off the wall on Covid. I don’t have any intelligent suggestions on this myself but one does faintly smell a rat.

    Reply
  11. Stillfeelinthebern

    My local public health officer says some vaccine is coming here in about 2 weeks.

    With Pfizer announcing its vaccine data suggests the shots may be 90 percent effective at preventing COVID-19, the Fond du Lac County Public Health Officer tells WFDL news Fond du Lac County will be receiving it’s first small batch of the vaccine before the end of the month. Public Health Officer Kim Mueller says the first doses will be distributed by the state Health Department to local hospitals or Public Health Departments in the next two weeks or less. Mueller says the initial amount will be small and will go to hospitals who will prioritize the small group of individuals who will be able to get it.

    https://www.radioplusinfo.com/2020/11/10/11-10-20-fdl-county-to-receive-first-small-batch-of-coronavirus-vaccine-before-the-end-of-the-month/

    Reply
  12. Blue Pilgrim

    A fleet of vans with -80 freezers on them can deliver vaccine like ice cream trucks deliver ice cream or reefer trucks distribute frozen foods (with daily deliveries), to local clinics. If the US government can have fleets of tanks and armored vehicles with machine guns and microwave crowd dispersal equipment then surely it can manage -80 refrigerator vans. These could be handy to have around for the next pandemic too.

    Reply
    1. Clive

      Nothing is impossible but do you know even the basics about ultra low temperature refrigeration? This ASHRAE note gives a high level overview https://mandtsystems.com/documents/ASHRAE_R02_39SI.pdf

      Read and learn — you need specific materials, components designed to work at very low temperatures reliably such as lubrication and compressors that won’t fry their windings. Reefers are only commonly commercially available for low temperature storage (-20°C) not these sorts of very low temperatures.

      And remember we are talking about medical-grade storage here. This means thermocouples (again, accurate at unusually low temperatures), data logging for accurate record keeping, alarms, tamper-proof security etc. etc. etc.

      An ice cream truck? Really?

      Reply
      1. Phacops

        Exactly. Durability of materials and control of temperature under cryogenic conditions is demanding. In designing cryogenic systems for the production of I-123 via a Xe-123 intermediate (from proton bombardment of 1-127) I was fortunate to have the assistance of a retired materials engineer from Argon Labs. I learned a lot about materials and processes from him!

        Reply
      2. John King

        Plus all low temp storage and transport has significant safety issues since dry ice or LN2 sublimate into either CO2 or N2 which displace O2. Containers can’t be air tight as they have to vent the sublimated gases which then pose a suffocation risk so the transport vehicle has to have a means of monitoring O2 levels and ensuring outside air flow. The larger the quantity being transported, the more safety and monitoring procedures you need to have in place.

        Plus objects cooled to these temps pose burn risk to exposed skin.

        In your example of a refer trailer, there would have to be a ventilation procedure and individual O2 monitoring to ensure no one suffocates when working at the far end of the trailer. Don’t forget the O2 deprivation is a tough safety problem since there aren’t any obvious warning signs and it impairs your cognitive functions which interferes in your ability to respond to the situation.

        Reply
      3. vlade

        The distribution, at least to the last mile can be handled by dewars (I believe it’s now actually pretty common). You don’t need temperature monitoring there, because it’s basically binary – the vaccine containers either are fully submerged in liquid nitrogen, or not, and it has a very great advantage of no moving parts, no power etc. – it’s very simple.

        But I don’t believe it’s a feasible way to do long-distance distribution for very large amounts.

        That said, I do believe that _animal_ vaccination has dealt with similar problems before (a lot of animal needing to be vaccinated), so something can be learned. Of course, there are crucial differences, like you get all the animals in at once, you can chose when to vaccinate, it’s not as important to get things right because some animal collateral damage is acceptable etc. etc.

        But looking at the cold chain for animal vaccination would be IMO definitely worth it. But no one pays me to do that, so I’ll leave it at that.

        Reply
  13. PlutoniumKun

    Just as a thought experiment, I was wondering if a workable approach would be to bring the people to the vaccine rather than vice versa.

    If, say you had a catchment of 1 million people, and decided to set up a single vaccination point, lets say a military base or vacated shopping mall with suitable facilities within an hours drive/bus for the catchment, and gave yourself 3-4 months to vaccinate the population, and you keep that base open for 15 hours a day (say, 7am to 10pm daily), with a strict appointment system, that would mean approximately 1000 people a day, or around 75 an hour. That would mean requiring a bus an hour coming and going, along with parking space for around 50 cars, with maybe one vehicle a minute arriving. This would be the traffic levels of a mid-sized retail outlet. You’d need a waiting area for perhaps 150 people. Assuming its a quick shot, not requiring medical tests or blood sampling, then maybe a dozen medical personnel with 20 or 30 support staff (2 shifts) could do the vaccination, i.e. around 250 staff in total assuming 7 days working.

    You would obviously require maybe 4-500 such centres in the US, or 50 or so in countries like the UK or France. So, that would be 1000 such centres for North America and Europe, each with the required low temperature storage and a direct daily supply link from the manufacturer.

    That doesn’t seem to me to be a logistical impossibility, although it would of course depend on how quickly the storage system required for each centre could be supplied and set up.

    Reply
    1. Halcyon (formerly AnonyMouse)

      I note that this is very different to the plan envisaged by the UK Government at present, which involves every local GP authority establishing their own facility to administer the vaccine.

      A total of ~1200 facilities administering ~1000 vaccines per week, which would enable the intended initial targets (over 65s) to be vaccinated over the course of around three months.

      https://www.theguardian.com/world/2020/nov/10/gps-in-england-will-scale-back-care-to-deliver-covid-vaccines

      We know that the current authorities are used to massively over-promising and under-delivering (moonshot testing, hundreds of thousands of antibody tests, the NHS contact tracing app that was months late) and botching the logistics for many of these projects (see previously), as well as lacking in attention to detail. I wonder if any consideration has been given to the apparent requirement to have a fleet of ~1200 (portable?) storage units for this ultracold vaccine.

      P.S. I hope no one minds me changing my moniker to something a bit less anonymous as long as I acknowledge it for a bit

      Reply
    2. Ignacio

      That will be a must. It will be needed a ‘summoning program’ detailed and careful to bring people to the vaccine.

      Reply
      1. John Anthony La Pietra

        Would they come in alphabetical order? If so, then by last name, first name, or city/town/village/hamlet/othello?

        Or by age? (Which order?) Birthday? Taxable income, or net worth? (Which order?) Political preference?

        Families/households all together? What if someone volunteers not to get vaccinated, or let’s someone from another dwelling unit take their place? (Are we going to need, oh, I don’t know, some sort of full list of who lives where as a reference?)

        All of which impertinence is pertinent to the issue of whether all the difficulties will be technical. . . .

        Reply
    1. MS Server

      It seems like this would be a great time for the Federal government to step in and purchase/procure the needed freezers.

      Reply
      1. Synoia

        Which administration would that be Trump’s or Biden’s? /s

        That would require the two to coordinate. Which looks unlikely.

        Setting aside our differences….

        Reply
    2. richard

      yes, and the rural vs. urban Divide and Conquer game played by u.s. elites gets a new lease on life. Poor thing is a mere 150 years old and a million times healthier than me.

      Reply
  14. Cas

    We also don’t know how effective this vaccine will be even if the temperature problem is resolved. The data from the clinical trials have not been released. Apologies if these links have appeared earlier, but I haven’t seen them.

    https://www.statnews.com/2020/11/10/the-story-of-mrna-how-a-once-dismissed-idea-became-a-leading-technology-in-the-covid-vaccine-race/

    This is a long read, as Yves would say, get a cup of coffee. As Lambert pointed out yesterday, no synthetic mRNA has been approved to date. This article tells the fascinating story of its development. Some excerpts:
    “In the natural world, the body relies on millions of tiny proteins to keep itself alive and healthy, and it uses mRNA to tell cells which proteins to make. If you could design your own mRNA, you could, in theory, hijack that process and create any protein you might desire — antibodies to vaccinate against infection, enzymes to reverse a rare disease, or growth agents to mend damaged heart tissue.
    “The stumbling block,… was that injecting synthetic mRNA typically led to that vexing immune response; the body sensed a chemical intruder, and went to war….
    “Every strand of mRNA is made up of four molecular building blocks called nucleosides. But in its altered, synthetic form, one of those building blocks, … was throwing everything off by signaling the immune system. So Karikó and Weissman simply subbed it out for a slightly tweaked version, creating a hybrid mRNA that could sneak its way into cells without alerting the body’s defenses.”
    So how has that turned out? This article from National Geographic presents some of the issues.

    https://www.nationalgeographic.com/science/2020/11/pfizer-biontech-interim-report-promising-but-lack-of-data-very-concerning/

    In a nutshell:
    “As a scientist, you’d want to see actual data from a clinical trial to really know what the actual interpretation of the results are,” says Aliasger K. Salem, chair of pharmaceutical sciences at the University of Iowa.
    Even non-scientists would like to see actual data.

    Reply
    1. reprobate

      Credit where credit is due! Yves quoted that Bloomberg section about mRNA treatments never having been used and therefore requiring years of monitoring 2x in posts before Lambert did.

      Reply
  15. Tim

    “The requirement for extremely cold temperatures is likely to cause spoilage of a lot of vaccine,”

    No. The requirement for extremely cold temperatures is likely to cause a high percentage of ineffective vaccinations to be given. Yeah, I know, US quality control is supposed to prevent this, but I don’t trust our competence especially at the last stop.

    So a false sense of security is provided to the population who decide to go back to normal full bore, causing a wave of coronavirus, causing loss of faith in the vaccine and skepticism towards future vaccines and we never get out of this pandemic.

    The good new is just because the vaccine wouldn’t be effective doesn’t mean it will kill us or turn us into zombies. (knock on wood).

    Reply
  16. Ignacio

    It is indeed a challenge to deliver an RNA vaccine worldwide. RNA is, by definition, unstable as it can be attacked by RNAses that are present everywhere (particularly in our skin and have to be managed with gloves and other protections) and that is why it must be stored in conditions that avoid its degradation. To be sure, the -80ºC temperature range is for assurance of long term storage but it strikes me that the manufacturer cannot apparently ensure RNAse-free storage, or is it by default that totally RNAse-free environment for storage cannot be guaranteed.

    Because -80ºC can be achieved in ultrafreezers and these are not good for transport ice carbon (may be even liquid nitrogen) must be used for transport that must be expedited, free of delays and ensuring no cold chain breaking from ultrafreezer to ultrafreezer.

    Ultrafreezers are abundant in the pharma industry and in biotech R&D installations. Hospitals too have ultrafreezers but almost certainly the existing units will be dedicated solely for the vaccine and new will have to be purchased.

    The number of sites for vaccine deployment will be very much reduced by the need to keep the product frozen up until shooting the vaccine and this will require very good organization and coordination efforts and will indeed limit the speed of vaccine delivery. This is one of the drawbacks of the new technology: there is no experience in mass deployment. Be sure this will be tried with essential workers/sectors that will serve to train for the most difficult grand scale deployment.

    Yet, what worries me the most about the Pfizer vaccine is that we are all hurrying about something about we still know as little as a simple statistical analysis done very shortly after the vaccination start in phase III and we have no idea if the protection level that has been seen so far will last longer than 3 months, not to mention longer times, or if it will fall precipitously in short time.

    Two other drawbacks are that two shoots are needed for full protection (well… 90%) in an interval of about two weeks (more deployment problems) and the side effects after shooting which are quite nasty (the younger the nastier).

    Reply
    1. vlade

      I agree with your penultimate paragraph. We’re hoping to deploy a massive vaccinatiton problem on something that hasn’t been really long-term tested. I see how there’s pressure to do that, but at the same time, if we get something wrong, the consequences could be worse than the problem we’re trying to solve – we just don’t know.

      A sister or my wife is now down with something that looks very much like CV. Except she’s pretty sure she had CV in April this year. In both cases, she got it from her daughter, who is also expriencing the same symptoms as in the spring.

      Neither of them was tested in the spring, so we don’t know for sure, but at the moment I’d put reinfection as quite possible for both of them – amongh other reasons, because with all that’s going on, very few people are actually getting flu so far this year.

      Reply
      1. Ignacio

        Yet we cannot extrapolate the duration of protection from previous infection to vaccine duration since these musn’t operate equally. Coronavirus are well known meddlers of the immune system while the vaccine is CoV-free and might give different results.

        Reply
  17. Cuibono

    so i doubt the plan is to solve this immediately for the developing world. I suspect that this will be distributed through hospitals to HCWs where these issues will not be nearly so challenging. Later, other vaccines that are more heat stable will come along presumably.

    Reply
  18. petal

    This just showed up in my inbox: Pfizer’s COVID-19 vaccine looks impressive, but Sanofi, J&J and Novavax shots eye a logistics edge

    “Pfizer’s COVID-19 vaccine this week set high efficacy expectations, but the vaccine has stringent storage needs and requires two doses, creating significant challenges for a global vaccination campaign. If they prove themselves in the clinic, later-arriving shots from the likes of Sanofi, J&J and Novavax may be better suited for global distribution.

    Johnson & Johnson’s vaccine is in testing as a one- and two-dose regimen, for instance, and is expected to be stable at refrigerated temperatures of 35.6 to 46.4 degrees Fahrenheit. When J&J started its phase 3 trial in September, the company said the candidate is “compatible with standard vaccine distribution channels and would not require new infrastructure to get it to the people who need it.” Now, J&J is performing studies to confirm that expectation, a spokesman said.

    Sanofi and GSK’s program, a two-dose recombinant protein vaccine with an adjuvant, can be stored between 35.6 to 46.4 degrees Fahrenheit, or in a doctor’s office or pharmacy, a spokeswoman said. While the partners aren’t expected to be the first to deliver doses, execs have said they expect to play an important role in the global immunization push.”

    Reply
  19. DaveC

    The question is whether, when or how much to invest in a new cryo infrastructure. Pfizer’s lack of data transparency muddles the question. Facing rapidly spreading pandemic, the lack of data on the vaccine causes at least anxiety, and paralyzes mass investment. Clearly the US has not conceived an infrastructure & logistical solution for mass cryo distribution of the Pfizer vaccine. Seems likely that will limit broad distribution for 10s of months, not a few months.

    Pfizer has a huge stake in releasing the 1st successful vaccination, because almost all the competing vaccines do not require a new expensive medical cold chain distribution infrastructure. It seems more likely than not Pfizer will not be successful providing the BNT162 vaccine widely. Which suggests that a massive public investment in a new cold chain distribution infrastructure is not appropriate.

    Random comment from this thread: Pfizer’s public statements lean as far away from transparency and as close as possible to marketing hype.

    Prediction: BNT162 will be available for a few, possibly the 1st vaccine available for non-experimental use. Probably very expensive. Pray that the cold chain distribution failures are transparent & don’t end up injected into patients. I doubt it will even get to widespread distribution in healthcare workers

    Reply
  20. chris

    Something I haven’t seen discussed here, or anywhere else yet, is while we think we know the shipping and handling constraints we don’t know who is going to do the shipping and handling. These aren’t bags of potato chips. This isn’t like Amazon shipping boxes all over.

    If we’re really going to mobilize a world wide response using this technology, and we’re going to deploy the vaccine in regions with vastly different climates and storage facilities even in first world countries…who is going to handle all this? I don’t mean the logistics supervisor, I mean who is going to put on the gloves and be certified as clean enough to touch the boxes? Do we even have enough cold gear for people to handle what they’ll need to handle? Do we have enough people who want that job?

    Clive is absolutely right to bring up the cold chain constraints and refer to the ASHRAE limits on these kind of processes. The other details that ASHRAE will point you to are discussed by organizations like AHRI… because it is difficult to maintain the equipment for cold chain storage too. And it is essential that such equipment be maintained or else you’re spoiling all that medicine.

    I am alarmed at how so many people who are promising this vaccine as a solution seem to be ignorant of the logistical challenges involved with every phase of deploying this vaccine. It’s going to take a lot of money to build the infrastructure necessary to manufacture, process, ship, receive, deploy, and administer this vaccine based on what’s been described. We can’t even get sufficient PPE to our teachers in the middle of a pandemic but we’re hand waving away these challenges?

    Reply
  21. Etrigan

    The comments on this and other similar articles here on nakedcapitalism are really invaluable. I’m putting together a supplementary outline of concerns at all points of what I’m calling the “pharm to arm” logistics chain using these articles, papers and citations, everyone’s comments.
    I’m sectioning it off into:
    -mRNA creation
    -dosage creation
    -transportation
    -refrigeration
    @refrigeration units
    @expense/scarcity
    @manufacturing/parts
    @health hazards
    @systemic risks: grid and electricity
    -unpacking
    -administering dosage
    -monitoring
    –government
    –unknowns
    with the assumption that categories like various systemic fragilities, overpromising, will be spread out over all the categories.

    If anyone has ideas about categories based on Yve’s article, let me know? I want to boil everything down to bullet points to help explain to friends and colleagues, who are REALLY optimistic and grasping at the Pfizer hopes right now and who won’t wade through information like you brilliant people have.

    Reply
    1. chris

      Why didn’t you post this comment in rhyme? If your moniker matches the demon, take the time :p

      I agree with your categories. And with the observation that so many are hoping this vaccine is an “easy button” of sorts. Similar to how “following the science” will fix everything. It is tragic how most people in our society don’t even have the imagination to conceive of how screwed we are and how much work there is to do to get past this.

      Reply
      1. Etrigan

        Don’t tempt me to rhyme, I haven’t the time!

        I have firsthand seen large-scale, serious local disasters handled with ingenuity and grace by – people- who just showed up and stayed committed. Wrenching proffered solutions to these problems out of the hands of business experts and handing them over to the drivers, medical technicians, nurses, electrical and cooling trades, and coordinators who would actually deal with everything is an actual avenue to solving these difficulties. And even more wild and radical, if everyday citizens were involved the local potholes (some possibly quite literal) could be accounted for.

        Reply

Leave a Reply

Your email address will not be published. Required fields are marked *